Not all records have the same fields

[CplRabbit]

Evening All, 

I'm building a data structure in airtable to represent a rather large tree style database. We're talking about 50k to 100k objects in the tree. 

I've got the heirachy view working quite nicely by uploading the components using the api, and then (again using the api) linking the objects with parent/child relationships. 

The next issue I have is some of the data the objects have. Most objects in the tree have a unique set of data. For example, one might have {colour, size and weight} while another might have {power, consumption}

Is there a good way of representing this? The cleanest way I found was by creating another table of this data and then linking it to the original objects, but with 650k data points, this doesn't seem feasible on a large scale.

I can add them as generic fields i.e. data01, make it a text type and then put the value in like "colour:Blue", but it's not ideal.

Let me know if you have any suggestions

Rabbit

[eva78mason]

Hey Rabbit,
Here are a few concise options for handling unique object data in Airtable:

1. Dynamic Fields with JSON: Store varied data types in a JSON object within each record.

2. Custom Attributes Table: Create a separate Attributes table and link attributes to your objects.

3. NoSQL Database: Integrate a scalable database like MongoDB for handling large, unstructured data.

4. API & Scripting: Use Airtable’s scripting or external APIs to dynamically manage your data.

Regards,
Eva M.
MyChart Wellstar

[Dimitris_Goudis]

Hey @CplRabbit, 

It's very interesting that you work with the hierarchy graphs on Airtable! How many levels do you show on the final image? 

Working for a VC company I tried the extension but the results showing more than 2 levels in the hierarchy weren't a good option so I built a script to auto-generate the result using the mermaid library. 

To solve this issue I would recommend creating a formula field that will check if the field color is not empty and return the value in there. The same formula can check also the other fields conditionally. On your graph use the new formula field 😉 

Please feel free to reach out to me or schedule a 30 mins call with no commitment, happy to help 🙂 
https://calendar.app.google/UXXGxNm1HHVT9B34A

Thanks,
Dimitris Goudis

[john589dane]

---

@CplRabbit wrote:

Evening All, 

I'm building a data structure in airtable to represent a rather large tree style database. We're talking about 50k to 100k objects in the tree. 

I've got the heirachy view working quite nicely by uploading the components using the api, and then (again using the api) linking the objects with parent/child relationships. 

The next issue I have is some of the data the objects have. Most objects in the tree have a unique set of data. For example, one might have {colour, size and weight} while another might have {power, consumption}

Is there a good way of representing this? The cleanest way I found was by creating another table of this data and then linking it to the original objects, but with 650k data points, this doesn't seem feasible on a large scale.

I can add them as generic fields i.e. data01, make it a text type and then put the value in like "colour:Blue", but it's not ideal.

Let me know if you have any suggestions

Rabbit

---

Hello @CplRabbit, 
Given your large-scale Airtable structure with unique data per object, a flexible approach could be using JSON or key-value pairs stored in a single text field for each object. This allows each object to have its specific attributes without creating additional tables. Snaptik


Best Regards,
John Dane