Which problem is this feature request solving?

This proposal addresses two concerns:

1. Some existing SDKs have their own structs to unmarshal API data, and these structs must sometimes be tagged with "json" tags to unmarshal fileds in REST reponses that do not comply with CQ's naming convention that "column names must contain only lower-case letters, numbers and underscores, and must start with a lower-case letter or underscore".
   As an example, OpenStack's APIs for VM instances return a complex struct that has some fields annotated with funny names such as json:"OS-SRV-USG:launched_at". CloudQuery panics when it encounters such fields (should it? maybe it's a bug?!?).
2. From the documentation, it looks like if you want to specify a type transformation for a field, you have to specify it in the Columns slice in the table definition, but maybe a declarative approach would be more lightweight?

Describe the solution you'd like

I propose to add a couple of transformers leveraging CQ-specific tags, so one can annotate a struct with JSON tags for the sake of unmarshalling REST API responses without having to worry abount CQ's naming convention on columns, and then add CQ-specific tags (e.g. cq-name) and a CloudQuery-SDK-provided NameTransformer to produce the desired name based on the cq-name tag.

Same for type, with cq-type and a TypeTransformer.

I've done it in my custom plugin (see below) and it seems to work like a charm.

In the table declaration you would use:

		Transform: transformers.TransformWithStruct(
			&MyThirdPartySDKProvidedAPIResponseStruct{},
			transformers.WithPrimaryKeys("ID"),
			transformers.WithNameTransformer(TagNameTransformer), // use cq-name tags to translate name
			transformers.WithTypeTransformer(TagTypeTransformer), // use cq-type tags to translate type
		),

The struct would be annotated like this:

type MyThirdPartySDKProvidedAPIResponseStruct struct {
	ID         string `json:"id"`
        // bla bla bla...
	LaunchedAt Time   `json:"OS-SRV-USG:launched_at" cq-name:"launched_at" cq-type:"timestamp"`
}

In order to support this without having to reinvent the wheel with every plugin, I would like to see a couple of functions added to the plugin SDK:

func TagNameTransformer(field reflect.StructField) (string, error) {
	if cq := field.Tag.Get("cq-name"); cq != "" {
		return cq, nil
	}
	return transformers.DefaultNameTransformer(field)
}

func TagTypeTransformer(field reflect.StructField) (schema.ValueType, error) {
	if cq := field.Tag.Get("cq-type"); cq != "" {
		switch cq {
		case "timestamp":
			return schema.TypeTimestamp, nil
		// TODO: add all other cases here
		}
	}
	return transformers.DefaultTypeTransformer(field)
}

Pull request (optional)

• I can submit a pull request