We have a couple hundred connector functions right now. We have an issue where we need to update functions that reference a specific column that we previously thought was unique. Now we have to go through every connector function to find references to that column so we can properly modify the functions. Being able to search the actual queries for text would make this much quicker. I can only see this getting worse as time goes on and more functions are developed. We are trying to be incredibly deliberate with what goes in to our data structure to avoid rework, but I can’t imagine this will be the last time we have this problem.
Hey Brian, couple questions on this:
- You say columns, so I assume this is SQL connectors only?
- Do you also need to duplicate the connector functions every time? Or are you able to modify the SQL query with the existing inputs/outputs? If so, what are you doing to those columns? Just adding new ones to the query?
- Sorry, yes, SQL connectors.
- Both - have definitely duplicated my fair share of functions in the past after having forgotten an input/output. In this specific case, we might be able to just modify the query, but I’d have to take a closer look at a few functions first. I’m thinking we will need an additional input on some of the functions though… To be more specific, we discovered that our ‘itemid’ column was actually not unique across sites, and the primary key was actually a combination of ‘itemid’ and ‘siteid’, so we need to rework to not return duplicate results where not expected. Some of the functions may just need siteID as a new input with a corresponding addition to any WHERE clauses, but I haven’t dug in yet.
And, just so I fully understand, the column names are not at all correlated with the names of the inputs/outputs that are exposed in the “Signature” column?
Correct. The columns may only be used in a WHERE clause to filter things down. As well, we often alias columns to slightly more descriptive names.