Calculation Significant Figures

Both variables are number inputs that get populated from a button trigger.

Whenever the trigger is executed, the number value spits out more significant figures than it has been set to.

Both Quality Index (Lower & Upper Limits) variables have been set to Decimal setting '1.00" for 2 significant figures, but the table record consistently stores 3 sig figs.

image

Thanks for any help!

image

image

Hey @gentleboyjones -

The widget property for precision just adjusts how values are presented within the app editor, but doesn’t actually impact how that data is stored in the table itself. So regardless of that setting, all numbers are stored to ~9 decimals. The tables UI shows all numbers to 3 decimals, but if you pulled table data from a csv export or over the API, you would see all of those decimal places.

There is a feature request in the works to be able to control the decimal places throughout the data pipeline. While this feels like it should be a super quick change, there is a fair bit of engineering work on the backend to make this possible. One option currently to force a consistent number of decimals is to store that value as text using the TOTEXT expression function and TEXTTONUMBER function (if you need to convert back to a number).

Thanks for the patience!
Pete

Thanks, Pete. I’ll keep that in the back of my mind. (y).

Hi Pete, working on splitting device output data text string into the test value and Unit of Measure, then storing the numerical test value as a number, but TEXTTONUMBER is dropping the trailing zeros which are (potentially) significant digits. Any suggestions?

@jmlowden, good question. Try appending the ROUND() function before TEXTTONUMBER() for the number if 0s you need. If it still drops the 0s let me know and I can do some sleuthing to see if there’s a trick.

Also seeing that when I write the numerical data variable to a table, it is only saving three decimal places…

UPDATE: which I realize Pete explained above, just missed it first pass - the csv download has the same digits as what I am able to see in Player, but still dropping the terminal zeroes

I added the ROUND() function a suggested and even tried adding +1 to the second term in the function, but the output remains the same.

@jmlowden, I checked in with our product team on this. This is a known gap, and we have several projects in motion to address this. No hard delivery date right now but it is something we’re aware of and working on.

Do you know if any progress was made on this feature?

Hi @landers -

Welcome to Tulip Community! :tada:

I am checking in on this with the Tulip Product team, and will follow up next week. Have a good weekend :slight_smile:

@landers @jmlowden

Here are my thoughts about that.

  • trailing Zeros: Any number has infinitely trailing zeros. If you save 0.123 but you want to display 0.12300 this is the exact same number. It is only a formatting thing and that is possible with Text.
    If you want to save this as text you can do something like:
    left(@Variable.no_1 + '00000', find('.', @Variable.no_1 + '') + 6)
    Where 6 is the precision +1
    You can’t save the number of zeros in a number type variable, because it saves the value and no formatting.
    Also keep in mind, that numbers can not be infinitely precise in computers. The available physical storage is limited and there is always an error in a later decimal place. This is due to saving long numbers in a binary system with limited space.
    E.g.: 0.2 + 0.1 = 0.30000000000000004 (e.g. in Tulip, Java Script…)

  • precision: I personally like to store the values as precise as available but only handle the way they are displayed. So in the Frontend, it will always be shown as 0.3 while the table saved 0.30000000000000004.
    If this is explicitly not wanted, simply always add a round()function to any number saving to a table…e.g.:
    image
    this will actually save 0.3 in the table …

So I think, if you think about, what you actually need, it is already possible to achieve that.
If a given precision of a measuring device is relevant, I save that as an Integer separately…

1 Like

Good idea, we have since used that approach to create record IDs with an ordinal numerical component:

Our earlier approach for significant digits could likely be simplified using this approach–this is the more complicated if/then/else approach currently in place:

I would imagine I could just reference a total precision variable calculated based on the specification precision in my LEFT() text function.