Great news - We’re excited to announce a much-requested platform update to how we store and preserve numeric data, which we’ll start rolling out as of r306/early February.
Details –
We’re updating how numbers are represented in Tulip Tables and Completions to support capturing, storing, and displaying trailing zeros, for example, 1.20 versus 1.2.
Number data, previously stored in our Postgres database as type Float, will be stored, moving forward, as a different type called Postgres Decimal, which is able to store numeric data with greater precision.
After release 306, we’ll start iteratively rolling out the following changes to instances, finalizing the rollout with LTS14.
Outcomes of this change include:
Operator data entered through Number Inputs will preserve and display up to 20 digits after the decimal, including trailing zeros. Inputs over 20 digits will be rounded to the nearest 20th digit. Previously we supported 15 digits of precision.
For numbers with many digits, Tulip Tables will now show all the available digits, up to Table column width. Previously Tulip Tables only displayed 3 digits after the decimal in the grid view. Selecting the Table Record will show the entire number, as before.
App Builders can now specify and enforce digits via:
On data input: A new Number Input validation rule for specifying the exact number of decimal places
For computation: Updated Expressions for managing decimal places of external inputs like Connectors, Devices, etc. and mathematical computations - ROUND and GETDECIMALPLACES.
One question though - this post says “Operator data entered through Number Inputs will preserve…” does this only work for the official number input?
I have some custom widgets to collect numbers with trailing zeros after a decimal to clarify precision of a measurement. The widgets use props to save the entered values into numeric app variables. Would trailing zeros passed from the custom widget directly to a numeric variable preserve the trailing zeros too?
Storing 1.00 and 1.0 differently is part of our efforts.
Does that mean Significant Figures is coming to Tulip?
To make things clear, please let me know how the new Numbers with Precision will calculate.
If we add 22.22 and 0.1 which would I get?
22.32
22.3
If we multiply 22.22 and 0.1 which would we get?
2.222
2
(If significant figures are coming to Tulip, my next question would be, how would we deal with the ambiguity of integer with trailing zeros. Significant figures - Wikipedia)
Hi @doneil, Custom Widget inputs do not currently capture trailing zeros and would not be updated to do so in this release. Custom Widgets and other external inputs like Devices, Connectors, etc. are out of scope. Our update is focusing on the Number Input and Tulip Table direct inputs.
The answer to your question would depend on how you’re passing trailing zeros today from Custom Widgets back to the App - are you using Strings/Text to do that?
Possibly easiest to chat live - if you’d like we’d be happy to set up a call!
If a 2.1 and 2.10 are stored differently. will an expression like if(2.1 = 2.10, true, false) deliver a true or a false?
I guess I know the answer, but this is a big deal and will cause some confusion …
Since mathematical operations are not sig fig aware, equality and set operations treat numbers as equal outside of their number of decimal places or precision.
In particular this means that if(2.1==2.10) will evaluate to true. We’ll update the example post to clarify, and also add a list of examples in the Knowledgebase article.
Great question - we’ve migrated previously stored Floats to a new Decimal column, which we’ll be using going forward - so we’d be comparing Decimal 2.1 to Decimal 2.10 - which would be equal.
In particular, the clarification here is that going forward, we’d be using Decimals both for newly stored values and to represent previously stored Floats.
Data collected prior to Release 306 has been completely preserved, keeping the numeric values the same. Any new incoming data is rounded to 20 digits after the decimal point.
For historical data, we migrated Floats to Decimal values in a way that preserved their decimal digits completely. This means their numeric value remains the same when viewed as text or exported in CSV - but the underlying type we’re using to store the numbers is now implemented by Postgres Decimal instead of Floats.
If I input “9.012345678901234567891” in the number input widget with the Maximal decimal places=20, I see “9.01234567890123456789”. So far so good.
If I input “9.012345678901234567899”, in the same widget, I still see the same “9.01234567890123456789”.
I was expecting to see “9.01234567890123456790” based on this article from the Knowledge Base.
@ta-aoki - The easy way to check if you have Numbers Precision is by checking whether you can save numbers with trailing zeros like 1.00 to Tables. Or please reach out to me in Direct Message or to Support with your instance URL, and we can confirm if you have Numbers Precision already enabled.
@thorsten.langner - (edited) let me look into this for you. You should be able to enter 20 digits after the decimal.
I think this answers some of the questions - let me know if any are left after the clarification in 2.
Hi Olga,
Thanks for clarifying.
But I think then the widget should not allow to set it to 20 now… the limit here switched from 16 to 20 as well.
I have no use case yet, but when I “allow” 21 digits it will reset to 20… if only 16 is allowed…?
@OlgaStroilova I apologize if I missed this being posted somewhere but just wanted to confirm that math with decimals has also been resolved in this or another recent update.
Specifically, I noticed a weird behavior 8 months ago (Support ticket # 19362) where a simple decimal number minus another decimal number would result in a wrong answer…consistently the same wrong answer. Example: 3.6715 – 3.6714 = 0.00009999999999976694 when clearly it should just equal 0.0001.
At the time I was told “This is a limitation of the underlying programming language, Javascript. In this (most used language in the world), if you try to perform that calculate you get that result. That result has 99.99999999976694% accuracy. Even if you performed billions of calculations on this, it would still be just as accurate.”
And so, it was - if I performed this math multiple time the answer would always be 0.00009999999999976694.
Yesterday I noticed this issue seems to be resolved - is that true?
previously math ran into floating point imprecision limitations, since we were storing numbers as Javascript/Postgres Floats (very commonly used type - that has some limitations including the one you outlined).
Yes, if you performed that operation repeatedly in the past, you’d get the same floating point math answer. Floating point inaccuracy is rare but not “random”.
Most importantly - Yes, this release would address this imprecision issue for Apps. The new underlying data type we’re using now, Postgres Decimal, does not have imprecision issues. Mathematical operations performed in Apps will be precise.
Caveat: we’re still working on moving Automations over to the new type - coming likely in mid 2025. For now Automations still perform floating point mathematical operations.
