Design spike: Brainstorm/research the correct level of customization for generic processors

[chowbao]

Brainstorm and research different customization options for CDP processors.

One of the main goals is to enable developers to access any low level field, without SDF still defining a data model (ie, payments)

There is a draft PR that relates this design spike. Note: placed randomly in stellar-etl for convenience

Considerations:

• What is the correct level of customization?
  • Call low level helper functions in new go function in ingest/application
  • Config/data model JSON files defining what data is wanted which then calls helper functions to call
  • Drag and drop UI
  • Something completely different or in between these options
• Maybe look into XDR <> JSON functions
• Needs to work for both Horizon and Hubble
  • Horizon and Hubble are our "real" world examples that we'd want to apply this to
• Type safety
• Are there any edge cases where this proposal won't work?
  • How much do these edge cases matter?
• Do any of these decisions affect ingest/cdp: Create low level helper functions for pulling out specific fields in LCM #5475?