UDT codec decode is too restrictive in decoding of unknown fields

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Duplicate
    • Priority: Normal
    • None
    • Component/s: Core
    • None
    • Low Hanging Fruit
    • User Report

      The current UDT code logic in driver 4.x expects the number of UDT fields exactly equal to the number of fields in the schema version used to create the codec.

      As a result it makes applications less friendly to a live upgrade logic when we change DB schema and do a rolling restart of applications to switch from an old app version to a new one. In this scenario we can add a new field to UDT (backward compatible change) and a new application version can write this UDT with a new field filled but the old application version will fail to decode such UDT with the extra unknown field.

      Driver 3.x UDT codec has a different behaviour - the UDT codec logic reads known UDT fields received from network and ignores the unknown tail, which makes in tolerant to such schema changes: https://github.com/apache/cassandra-java-driver/blob/3.x/driver-core/src/main/java/com/datastax/driver/core/TypeCodec.java#L2202

      A code example to illustrate the issue:

      Schema

      CREATE TYPE test_keyspace.basicInfoInner (
        weight text,
        height text
      );
      
      CREATE TYPE test_keyspace.basicInfo (
        birthday    timestamp,
        nationality text,
        inner       frozen<basicInfoInner>
      );
      
      CREATE TABLE test_keyspace.tableWithUdt
      (
          id    varchar,
          value frozen<list<basicInfo>>,
          PRIMARY KEY (id)
      ); 

      Code:

      CqlSession session = getSession();
      UdtCodec udtCodec = new UdtCodec(userDefinedType);
      TypeCodec<List<UdtValue>> listOfUdtCodec = TypeCodecs.listOf(udtCodec);
      
      {
          ResultSet resultSet = session.execute("select id, value from tableWithUdt");
          for (Row row : resultSet) {
              String id = row.getString(0);
              List<UdtValue> udtValueList = row.get(1, listOfUdtCodec);
          }
      }
      
      session.execute("ALTER TYPE basicInfo ADD details text"); 
      
      session.execute("INSERT INTO tableWithUdt (id, value) VALUES ('4', [\n" +
              " { birthday : '1993-06-18', " +
              "   nationality : 'New Zealand', " +
              "   inner: { weight : '70', height : '70' } " +
              " },\n" +
              " { birthday : '1993-06-19', " +
              "   nationality : null, " +
              "   inner: { weight : '70', height : null }, " +
              "   details: 'added details'" + // <=== insert a value for the new added UDT field
              " }\n" +
              "  ])");
      
      {
          ResultSet resultSet = session.execute("select id, value from tableWithUdt");
          for (Row row : resultSet) {
              String id = row.getString(0);
              //  ==== UdtCodec throws IllegalArgumentException: Too many fields in encoded UDT value, expected 3 ==== 
              List<UdtValue> udtValueList = row.get(1, listOfUdtCodec);
          }
      }
      
      private static UserDefinedType getUserDefinedType(CqlSession session, String udtName) {
          return session.getMetadata()
                  .getKeyspace(session.getKeyspace().orElseThrow(() -> new IllegalArgumentException("Udt name " + udtName + " can't be loaded due to null keyspace")))
                  .orElseThrow(() -> new IllegalArgumentException("Metadata is not found"))
                  .getUserDefinedType(udtName)
                  .orElseThrow(() -> new IllegalArgumentException("Udt name " + udtName + " isn't found"));
      }
      

            Assignee:
            Dmitry Konstantinov
            Reporter:
            Dmitry Konstantinov
            Dmitry Konstantinov
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 50m
                50m