PACKおよびUNPACKの実装の詳細

のバイト操作セクションマイケルソンの白い文書には、次のように記載されています:

「PACK:データの一部を最適化されたバイナリ表現にシリアル化します.」

PACKとUNPACKがどのように実装されているか、およびバイナリ表現とは何かについて詳しく知りたいです.ソースコードを調べましたが、実装にコメントはあまりありません.

編集:

ユーザーcatsigmaが提供した詳細は非常に役立ちますが、ソースコードでシリアル化がどのように機能し、バイナリコードマッピングへの操作がどこから来るのかを追跡できるようにしたいと思います.

ここでinterp関数とPackの実装を見つけました:

src/proto_alpha/lib_protocol/src/script_interpreter.ml#L697

 let rec interp =
...
| Pack t, Item (value, rest) ->
    Script_ir_translator.pack_data ctxt t value >>=? fun (bytes, ctxt) ->
    logged_return (Item (bytes, rest), ctxt)
 

pack_dataはここで定義されています:

src/proto_alpha/lib_protocol/src/script_ir_translator.ml#L3162

 let pack_data ctxt typ data =
  unparse_data ctxt Optimized typ data >>=? fun (data, ctxt) ->
  let unparsed = strip_annotations @@ data in
  let bytes = Data_encoding.Binary.to_bytes_exn expr_encoding (Micheline.strip_locations unparsed) in
  Lwt.return @@ Gas.consume ctxt (Script.serialized_cost bytes) >>=? fun ctxt ->
  let bytes = MBytes.concat "" [ MBytes.of_string "\005" ; bytes ] in
  Lwt.return @@ Gas.consume ctxt (Script.serialized_cost bytes) >>=? fun ctxt ->
  return (bytes, ctxt)
 

Data_encoding.Binary.to_bytes_exnは主要な関数のように見えます.

src/lib_data_encoding/binary_writer.ml#L338

 let to_bytes_exn e v =
  match Encoding.classify e with
  | `Fixed n -> begin
      (* Preallocate the complete buffer *)
      let state = { buffer = MBytes.create n ;
                    offset = 0 ; allowed_bytes = Some n } in
      write_rec e state v ;
      state.buffer
    end
  | `Dynamic | `Variable ->
      (* Preallocate a minimal buffer and let's not hardcode a
         limit to its extension. *)
      let state = { buffer = MBytes.create 4096 ;
                    offset = 0 ; allowed_bytes = None } in
      write_rec e state v ;
      MBytes.sub state.buffer 0 state.offset
 

それでは、Falseのエンコーディングがどうなるかを調べてみましょう. src/lib_data_encoding/binary_writer.ml <を再編成したコードを次に示します./a>:

 let rec write_rec : type a. a Encoding.t -> state -> a -> unit =
  fun e state value ->
    let open Encoding in
    match e.encoding with
    | Bool -> Atom.bool state value
    ...

module Atom = struct
  let bool state v = uint8 state (if v then 255 else 0)
  let uint8 = int `Uint8
  let int kind state v =
    check_int_range (Binary_size.min_int kind) v (Binary_size.max_int kind) ;
    let ofs = state.offset in
    may_resize state (Binary_size.integer_to_size kind) ;
    set_int kind state.buffer ofs v

  let check_int_range min v max =
    if (v < min || max < v) then
      raise (Invalid_int { min ; v ; max })

  let set_int kind buffer ofs v =
    match kind with
    ...
    | `Uint8 -> MBytes.set_int8 buffer ofs v
end
 

src/lib_protocol_environment/sigs/v1/mBytes.mli

mBytesは外部ライブラリのようですが、名前が見つかりません.また、これがtezbridge-cryptoで説明されているop_mappingとどのように一致するかわかりません.

In the Operations on bytes section of the Michelson white document, it is mentioned that:

"PACK: Serializes a piece of data to its optimized binary representation."

I am curious to know more details of how PACK and UNPACK are implemented and what the binary representation is. I have looked through the source code and there are not any many comments in the implementation.

Edit:

The details that user catsigma have given are very useful, but I would still like to be able to follow how serialization works in the source code and where the operation to binary code mappings come from.

I found the interp function here and the implementation of Pack:

src/proto_alpha/lib_protocol/src/script_interpreter.ml#L697

let rec interp =
...
| Pack t, Item (value, rest) ->
    Script_ir_translator.pack_data ctxt t value >>=? fun (bytes, ctxt) ->
    logged_return (Item (bytes, rest), ctxt)

pack_data is defined here:

src/proto_alpha/lib_protocol/src/script_ir_translator.ml#L3162

let pack_data ctxt typ data =
  unparse_data ctxt Optimized typ data >>=? fun (data, ctxt) ->
  let unparsed = strip_annotations @@ data in
  let bytes = Data_encoding.Binary.to_bytes_exn expr_encoding (Micheline.strip_locations unparsed) in
  Lwt.return @@ Gas.consume ctxt (Script.serialized_cost bytes) >>=? fun ctxt ->
  let bytes = MBytes.concat "" [ MBytes.of_string "\005" ; bytes ] in
  Lwt.return @@ Gas.consume ctxt (Script.serialized_cost bytes) >>=? fun ctxt ->
  return (bytes, ctxt)

Data_encoding.Binary.to_bytes_exn looks like the key function.

src/lib_data_encoding/binary_writer.ml#L338

let to_bytes_exn e v =
  match Encoding.classify e with
  | `Fixed n -> begin
      (* Preallocate the complete buffer *)
      let state = { buffer = MBytes.create n ;
                    offset = 0 ; allowed_bytes = Some n } in
      write_rec e state v ;
      state.buffer
    end
  | `Dynamic | `Variable ->
      (* Preallocate a minimal buffer and let's not hardcode a
         limit to its extension. *)
      let state = { buffer = MBytes.create 4096 ;
                    offset = 0 ; allowed_bytes = None } in
      write_rec e state v ;
      MBytes.sub state.buffer 0 state.offset

Now let's try to find what the encoding of False would be. Here is some more code reorganized src/lib_data_encoding/binary_writer.ml:

let rec write_rec : type a. a Encoding.t -> state -> a -> unit =
  fun e state value ->
    let open Encoding in
    match e.encoding with
    | Bool -> Atom.bool state value
    ...

module Atom = struct
  let bool state v = uint8 state (if v then 255 else 0)
  let uint8 = int `Uint8
  let int kind state v =
    check_int_range (Binary_size.min_int kind) v (Binary_size.max_int kind) ;
    let ofs = state.offset in
    may_resize state (Binary_size.integer_to_size kind) ;
    set_int kind state.buffer ofs v

  let check_int_range min v max =
    if (v < min || max < v) then
      raise (Invalid_int { min ; v ; max })

  let set_int kind buffer ofs v =
    match kind with
    ...
    | `Uint8 -> MBytes.set_int8 buffer ofs v
end

Now we need to find MBytes.set_int8. I have only found the type signature file.

src/lib_protocol_environment/sigs/v1/mBytes.mli

mBytes looks like it is an external library but I haven't found the name. Also, I am not seeing how this matches up with the op_mapping as mentioned in tezbridge-crypto.