2007年04月28日 02:30 [Edit]

CPAN - URI::Escape::XS Released!

cpan

URI-Escape-XS を Release したのでお知らせします。


なぜ車輪を再発明したかといえば、今ある車輪のころがりがよくなかったから。

URI::Escapeへの不満は二つあって、一つは速度が不十分だったこと。LWPなどと組み合わせて使う場合は、正規表現ベースの変換でも、他のタスクの方がずっと重いので充分速いのですが、ログの解析する時などに利用すると、ずいぶんと遅く感じます。このあたりはある作業をしていて、気になってProfileしてみてはじめて気がつきました。

もう一つは、%uHHHHの対応。一応にぽたん作のURI::Escape::JavaScriptというものがあるのですが、コードを見るとSurrogate Pairに未対応。このあたりの問題は

でも触れたのですが、丸い車輪が見当たらない。しばらくは「とりあえずいっか。どうせ%uHHHHなんて規格外だし」とたかをくくっていたのですが、先日ある仕事で切実に動く奴が必要になったのでえいやと作りました。

詳しくはPODを参照のこと。

Enjoy!

Dan the Perl Monger

NAME
    URI::Escape::XS - Drop-In replacement for URI::Escape

VERSION
    $Id: XS.pm,v 0.1 2007/04/27 17:17:46 dankogai Exp dankogai $

SYNOPSIS
        # use it instead of URI::Escape
        use URI::Escape::XS qw/uri_escape uri_unescape/;
        $safe = uri_escape("10% is enough\n");
        $verysafe = uri_escape("foo", "\0-;\377");
        $str  = uri_unescape($safe);

        # or use encodeURIComponent and decodeURIComponent
        use URI::Escape::XS;
        $safe = encodeURIComponent("10% is enough\n");
        $str  = decodeURIComponent("10%25%20is%20enough%0A");

EXPORT
  by default
    "encodeURIComponent" and "decodeURIComponent"

  on demand
    "uri_escape" and "uri_unescape"

FUNCTIONS
  encodeURIComponent
    Does what JavaScript's encodeURIComponent does.

      $uri = encodeURIComponent("http://www.example.com/");
      # http%3A%2F%2Fwww.example.com%2F

    Note you cannot customize characters to escape. If you need to do so,
    use "uri_escape".

  decodeURIComponent
    Does what JavaScript's decodeURIComponent does.

      $str = decodeURIComponent("http%3A%2F%2Fwww.example.com%2F");
      # http://www.example.com/

    It decode not only %HH sequences but also %uHHHH sequences, with
    surrogate pairs correctly decoded.

      $str = decodeURIComponent("%uD869%uDEB2%u5F3E%u0061");
      # \x{2A6B2}\x{5F3E}a

    This function UNCONDITIONALLY returns the decoded string with utf8 flag
    off. To get utf8-decoded string, use Encode and

      decode_utf8(decodeURIComponent($uri));

    This is the correct behavior because you cannnot tell if the decoded
    string actually contains UTF-8 decoded string, like ISO-8859-1 and
    Shift_JIS.

  uri_escape
    Does exactly the same as URI::Escape::uri_escape() except when
    utf8-flagged string is fed.

    URI::Escape::uri_escape() croak and urge you to "uri_escape_utf8()" but
    it is pointless because URI itself has no such things as utf8 flag. The
    function in this module ALWAYS TREATS the string as byte sequence. That
    way you can safely use this function without worring about utf8 flags.

    Note this function is NOT EXPORTED by default. That way you can use
    URI::Escape and URI::Escape::XS simultaneously.

  uri_unescape
    Does exactly the same as URI::Escape::uri_escape() except when %uHHHH is
    fed.

    URI::Escape::uri_unescape() simply ignores %uHHHH sequences while the
    function in this module does decode it into the corresponding UTF-8 byte
    sequence.

    Like uri_escape, this funciton is NOT EXPORTED by default.

  Note on the %uHHHH sequence
    With this module the resulting strings never have the utf8 flag on. So
    if you want to decode it to perl utf8, You have to explicitly decode via
    Encode. Remember. URIs have always been a byte sequence, not UTF-8
    characters.

    If %uHHHH sequence became standard, you could have safely told if a given
    URI is in Unicode. But more fortunately than unfortunately, the RFC
    proposal was rejected so you cannnot tell which encoding is used just by
    looking at the URI.

    <http://en.wikipedia.org/wiki/Percent-encoding#Non-standard_implementations>

    I said fortunately because %uHHHH can be nasty for non-BMP characters.
    Since each %uHHHH can hold one 16-bit value, you need a *surrogate pair*
    to represent it if it is U+10000 and above.

    In spite of that, there are a significant number of URIs with %uHHHH
    escapes. Therefore this module supports decoding only.

SPEED
    Since this module uses XS, it is really fast except for
    uri_escape("noop").

    Regexp which is used in URI::Escape is really fast for non-matching but
    slows down significantly when it has to replace string.

  BENCHMARK
    On Macbook Pro 2GHz, Perl 5.8.8.

     http://www.google.co.jp/search?q=%E5%B0%8F%E9%A3%BC%E5%BC%BE
     ============================================================
     Unescape it
     -----------
     U::E      58526/s       --     -88%
     U::E::XS 486968/s     732%       --
     --------------
     Escape it back
     --------------
     U::E      30046/s       --     -78%
     U::E::XS 136992/s     356%       --

     www.example.com
     ===============
     Unescape it
     -----------
                   Rate     U::E U::E::XS
      U::E     821972/s       --      -4%
      U::E::XS 854732/s       4%       --
     --------------
     Escape it back
     -------------
     U::E::XS 522969/s       --      -7%
     U::E     565112/s       8%       --

AUTHOR
    Dan Kogai, "<dankogai at dan.co.jp>"

BUGS
    Please report any bugs or feature requests to "bug-uri-escape-xs at
    rt.cpan.org", or through the web interface at
    <http://rt.cpan.org/NoAuth/ReportBug.html?Queue=URI-Escape-XS>. I will
    be notified, and then you'll automatically be notified of progress on
    your bug as I make changes.

SUPPORT
    You can find documentation for this module with the perldoc command.

        perldoc URI::Escape::XS

    You can also look for information at:

    * AnnoCPAN: Annotated CPAN documentation
        <http://annocpan.org/dist/URI-Escape-XS>

    * CPAN Ratings
        <http://cpanratings.perl.org/d/URI-Escape-XS>

    * RT: CPAN's request tracker
        <http://rt.cpan.org/NoAuth/Bugs.html?Dist=URI-Escape-XS>

    * Search CPAN
        <http://search.cpan.org/dist/URI-Escape-XS>

ACKNOWLEDGEMENTS
    Gisle Aas for URI::Escape

    Koichi Taniguchi for URI::Escape::JavaScript

COPYRIGHT & LICENSE
    Copyright 2007 Dan Kogai, all rights reserved.

    This program is free software; you can redistribute it and/or modify it
    under the same terms as Perl itself.


この記事へのトラックバックURL

この記事へのコメント
通りすがりさん、
お、さくさく感が戻ってきました。
他にもcan'tやcould'veがハイライトをおかしくしていたので、
手元のRepositoryでもこちらのREADMEの移しも直しておきました。
Google Prettyprintの意外な効用。
Dan the Pretty Printed
Posted by at 2007年04月28日 15:06
blogに貼り付けてるPODのSYNOPSISの項のサンプルで
ダブルクォートの閉じ忘れでハイライトがおかしくない?
Posted by 通りすがり at 2007年04月28日 13:31