2

我必须实现一些专有的二进制格式,并想用 scodec 来做到这一点。但是,我找不到简洁的解决方案。格式如下: 一个文件由多个 Records 组成,其中每个 Record 都以一个 little endian 16 位数字“t”(uint16L) 为前缀。根据 t 的第一个和第二个字节的值,记录可以分为 4 类:

  • 正常:t.first != 0 && t.second == 0
  • 条形图:t.first == 0x08 && t.second == 0xFF
  • Foo: t.first == 0x04 && t.second == 0x05
  • 无效:t 不是以上任何一项

如果 t 无效,则程序应退出,因为文件已损坏。如果 t 是 Normal 或 Bar,则 Record 的长度遵循 32 位 little endian int。如果 t 为 Foo,则必须先解析另一个 16 位大端int,然后才能将长度解析为 32 位 BE int。

- Normal: ("t" | uint16L) :: ("length" | uint32L) :: [Record data discriminated by t]
- Bar: ("t" | constant(0x08FF)) ::  ("length" | uint32L) :: [Record data of Bar]
- Foo: ("t" | constant(0x0405)) :: uint16 :: ("length" | uint32) :: [Record data of foo]
- Invalid: ("t" | uint16L ) :: fail(Err(s"invalid type: $t"))

此外,“Normal”中 t 的某些值未使用,应生成 UnknownRecord(类似于此处的 mpeg 实现:https ://github.com/scodec/scodec-protocols/blob/series/1.0.x/src/main /scala/scodec/protocols/mpeg/Descriptor.scala )

这是我目前的方法,但感觉不太清楚,我觉得我更多地围绕 scodec 工作,而不是使用它。有任何想法吗?随意在下面废弃我的代码..

sealed trait ContainerType
object ContainerType{
    implicit class SplitInt(val self: Int) extends AnyVal{
        def first = self & 0xFF
        def second = (self >> 8) & 0xFF
    }

    case object Normal extends ContainerType
    case object Bar extends ContainerType
    case object Foo extends ContainerType
    case object Invalid extends ContainerType

    val codec: Codec[ContainerType] = {
        def to(value: Int): ContainerType = value match{
            case v if value.first != 0 && value.second == 0 => Normal
            case v if value.first == 0x08 && value.second == 0xFF => Bar
            case v if value.first == 0x04 && value.second == 0x05 => Foo
            case _ => Invalid
        }

        uint16L.xmap(to, ??) // don't have value here
        // if I use case classes and save the value I can't discriminate by it in RecordPrefix
    }        
}

sealed trait RecordPrefix{
    def t : Int,
    def length: Int
}
object RecordPrefix {

    case class Normal( override val t: Int, override val length: Int) extends RecordPrefix
    object Normal{
        val codec: Codec[Normal] = ??
    }
    case class Bar(override val t: Int, override val length: Int) extends RecordPrefix
    object Bar{
        val codec: Codec[Bar] = ??
    }
    case class Foo(override val t: Int, foobar: Int, length: Int) extends RecordPrefix
    object Foo{
        val codec: Codec[Foo] = ??
    }


    val codec: Codec[RecordPrefix] = {
        discriminated[RecordPrefix].by(ContainerType.codec)
        .typecase(Normal, Normal.codec)
        .typecase(Bar, Bar.codec)
        .typecase(Foo, Foo.codec)
        // how to handle invalid case ?
    }

}

case class Record(prefix: RecordPrefix, body: RecordBody)

sealed trait RecordBody
//.... How can I implement the codecs?

PS:这是我在这里的第一个问题,我希望它足够清楚。=)

Edit1:我找到了一个至少可以完成这项工作的实现。如果记录未知,我会权衡以再次检查条件,以获得更清晰的层次结构。

trait KnownRecord
sealed trait NormalRecord extends KnownRecord

case class BarRecord(length: Int, ..,) extends KnownRecord
object BarRecord {
    val codec: Codec[BarRecord] = {
        ("Length" | int32L) ::
        //...
    }.as[BarRecord]

}

case class FooRecord(...) extends KnownRecord
object FooRecord {
    val codec: Codec[FooRecord] = // analogue
}

case class A() extends NormalRecord
case class B() extends NormalRecord
// ...

case class UnknownRecord(rtype: Int, length: Int, data: ByteVector)
object UnknownRecord{

    val codec: Codec[UnknownRecord] = {
        ("Type" | Record.validTypeCodec) ::
        (("Length" | int32L) >>:~ { length =>
            ("Data" | bytes(length - 6)).hlist
        })
    }.as[UnknownRecord]
}

object Record{
    type Record = Either[UnknownRecord, KnownRecord]

    val validTypeCodec: Codec[Int] = {
        uint16L.consume[Int] { rtype =>
            val first = rtype & 0xFF
            val second = (rtype >> 8) & 0xFF
            rtype match {
                case i if first != 0 && second == 0 => provide(i)
                case i if first == 0x04 && second == 0x05 => provide(i)
                case i if first == 0xFF && second == 0x08 => provide(i)
                case _ => fail(Err(s"Invalid Type: $rtype!"))
            }
        } (identity)
    }

    def normalCodec(rtype: Int): Codec[NormalRecord] = {
        discriminated[NormalRecord].by(provide(rtype))
        .typecase(1, A.codec)
        .typecase(2, B.codec)
        .typecase(3, C.codec)
        .typecase(4, D.codec)
        .framing(new CodecTransformation {
            def apply[X](c: Codec[X]) = variableSizeBytes(int32L, c.complete,
                                                          sizePadding=6)
        })
    }.as[NormalRecord]


    val knownCodec: Codec[KnownRecord] = {
        val b = discriminated[KnownRecord].by(("Type" | uint16L))
            .typecase(0x0504, FooRecord.codec)
            .typecase(0x08FF, BarRecord.codec)
        (1 to 0xFF).foldLeft(b) {
            (acc, x) => acc.typecase(x, normalCodec(x))
        }
    }

    implicit val codec: Codec[Record] = {
        discriminatorFallback(UnknownRecord.codec, knownCodec)
    }

Edit2:我在下面发布了一个替代解决方案作为答案

4

2 回答 2

2

我将其发布为答案,因为我对此解决方案感到满意,尽管这可能是我的第一个解决方案(问题中的 edit1)和这个解决方案之间的个人偏好问题。Shasticks 的回答也提供了一种有用的方法,如果人们想跟踪鉴别器的值(我宁愿不这样做)。

我希望这对其他人也有帮助。

这是解决方案 2:我没有使用预定义的编解码器,而是分别解码和编码。

decode 选择正确的编解码器而不对类型进行多次解码,而 encode 从 Recordtype 推导出正确的类型值(Bar/Foo -Records 具有常量类型,NormalRecords 由 Record.normalCodec 中的 DiscriminatorCodec 编码)

trait KnownRecord
sealed trait NormalRecord extends KnownRecord

case class BarRecord(..,) extends KnownRecord
object BarRecord {
    val codec: Codec[BarRecord] = {
        //...
    }.as[BarRecord]

}

case class FooRecord(...) extends KnownRecord
object FooRecord {
    val codec: Codec[FooRecord] = // ...
}

case class A() extends NormalRecord
case class B() extends NormalRecord
// ...

case class UnknownRecord(rtype: Int, length: Int, data: ByteVector)
object UnknownRecord{

    val codec: Codec[UnknownRecord] = {
        ("Type" | uint16L) ::
        (("Length" | int32L) >>:~ { length =>
            ("Data" | bytes(length - 6)).hlist
        })
    }.as[UnknownRecord]
}

sealed trait ContainerType
object ContainerType{

    case object FooType extends ContainerType
    case object BarType extends ContainerType
    case class NormalType(rtype: Int) extends ContainerType
    case class Invalid(rtype: Int) extends ContainerType

    implicit val codec: Codec[ContainerType] = {
        def from(value: Int): ContainerType = {
            val first = value & 0xFF
            val second = (value >> 8) & 0xFF
            value match {
                case 0x0504 =>  FooType
                case 0x08FF => BarType
                case i if (second == 0 && first != 0) => NormalType(i)
                case other => Invalid(other)
            }
        }

        def to(ct: ContainerType): Int = ct match {
            case FooType => 0x0302
            case BarType => 0x0FFF
            case NormalType(i) => i
            case Invalid(i) => i
        }

        uint16L.xmap(from, to)
    }

}

object Record{
    type Record = Either[UnknownRecord, KnownRecord]

    val ensureSize = new CodecTransformation {
        def apply[X](c: Codec[X]) = variableSizeBytes(int32L, c.complete,
                                                      sizePadding=6)
    }

    val normalCodec: Codec[NormalRecord] =
        normalCodec(uint16L).framing(ensureSize).as[NormalRecord]

    def normalCodec(discr: Codec[Int]) =
        discriminated[NormalRecord].by(discr)
        .typecase(1, A.codec)
        .typecase(2, B.codec)
        .typecase(3, C.codec)
        .typecase(4, D.codec)

    val knownCodec: Codec[KnownRecord] = {
        import ContainerType._

        def decodeRecord(bits: BitVector): Attempt[DecodeResult[KnownRecord]] =
        for {
            ct <- ContainerType.codec.decode(bits)
            rec <- ct.value match {
                case FooType => FooRecord.codec.decode(ct.remainder)
                case BarType =>
                    ensureSize(BarRecord.codec).decode(ct.remainder)
                case NormalType(i) =>
                    ensureSize(normalCodec(provide(i))).decode(ct.remainder)
                case Invalid(rtype) =>
                    Attempt.failure(Err(s"Invalid Type: $rtype!"))
            }
        } yield rec


        def encodeRecord(rec: KnownRecord): Attempt[BitVector] =
        rec match {
            case c: NormalRecord => normalCodec.encode(c)

            case fr: FooRecord => for {
                rtype <- ContainerType.codec.encode(FooType)
                record <- FooRecord.codec.encode(fr)
            } yield rtype ++ record

            case br: BarRecord => for {
                rtype <- ContainerType.codec.encode(BarType)
                record <- BarRecord.codec.encode(br)
                length <- int32L.encode((record.size / 8).toInt + 6)
            } yield rtype ++ length ++ record
        }

        Codec(Encoder(encodeRecord _), Decoder(decodeRecord _))
    }


    implicit val codec: Codec[Record] = {
        discriminatorFallback(UnknownRecord.codec, knownCodec)
    }
于 2016-11-15T16:06:28.240 回答
0

我不确定您尝试做的事情是否可以使用 a DiscriminatorCodec(如果我理解得很好,您想跟踪定义 uint16 的值ContainerType

一个选项可能是从 a 开始Codec[(Int, ContainerType)]并使用consume()它来选择基于ContainerType.

这可能看起来像:

def typeCodec: Codec[(Int, ContainerType)] = 
    uint16L.xmap(t => (t, to(t)), tup => tup._1)

/** Codec for decoding/encoding RecordPrefixes */
def prefixCodec: Codec[RecordPrefix] = 
    typeCodec.consume(selectCodec)(_.typeTuple)

def selectCodec(tup: (Int, ContainerType)) =
    tup match {
         case (i, Normal) => // Return codec for normal, using 'i' if required
         case ... 
    }

这将需要添加def typeTuple: (Int, ContainerType)功能RecordPrefix才能进行编码。

然后,一旦你有了你的,Codec[RecordPrefix]你就可以用它来解析其余的(使用consume(), flatPrepend(), ...)


或者,您可以peek(uint16L) :: <DiscriminatorCodec>在仍然使用鉴别器编解码器的同时保留该值。peek在这种情况下,编码时要小心。

希望有帮助。

于 2016-11-03T23:32:22.163 回答