We are planning to create an app that can capture a picture of an analog gage and send it to a Web API that returns the value of current needle’s position.
But Image input widget can only handle “image URL” only. On the other hand, Vision Camera widget can handle Base64String as a variable. There is one problem. Vision Camera only supports Windows camera only.
So, my suggestion is to handle Base64String variable with Image input widget. If it works, we can easily send an image to some Web APIs.
This is a great idea, I ran into this limitation just the other day, you hit on the way I got around this, with vision. I will bring this one to the engineering team to see if we can sneak it in sooner rather than later.
Just to add in to that. Can tulip handle a long Base64String for input to a variable. More specifically, to a Prop, or embedded in a custom widget?? I am thinking of specifically embedding sounds converted from mp3 to base64.
I have a custom widget in the works that populates a report with values passed on props (imagine a tulip app filling out an existing audit form). I passed this base64 out of my Custom Widget as a base64 string. The max string size tables will support is something like 2gb, so absolutely massive
Important to know - text fields have indexes on them in tables that limit their size more significantly to improve query performance, so if you want to store long base64 strings to tables, use image fields in tables.
Hi
I have a similar use case + one where I need to evaluate the output of a barcode quality tester. It is quite old so I wanted to document the output by capturing a image and pass the result through the Google vision API as Google connector is working flawlessly and now I was looking for a expression component to convert the image stored to base64. I don’t think an expression exist for this conversion.
Could you show me the widget you built, Pete?
Regards
Asger
To follow up on my previous post I put together a custom widget which converts an image to base64 (to subsequently allow me to send it through a Google OCR API).
Working with image from the web seems to work fine (both within app and in test mode)
Working with images stored within the instance database appear not to work, and I am unsure if e.g. the custom widget are missing some sort of access to the DB, or if I am missing some javascript code piece
When you say database, do you mean tables? if so, base64 must be stored to image fields in tables for a few technical reasons. While the underlying datatype for tables is text (in postgres) so they can support up to 2gb’s of text, the indexes that are used to drastically improve the performance of analytics are a lot more limited on the length of strings that can be stored in table text fields. I also ran into this issue.
I have made a custom widget for a specific usecase that I think can help both of you -
The usecase was a situation where a user wanted to populate an existing form with information from a tulip app and store that to a table. Maybe you have a standard audit form that you want Tulip to populate or a standard label you want to generate with a Tulip app
Additionally. I wanted this to be flexible to any number of inputs and any text positions, so you could use the same widget for multiple different forms.
This is what I came up with-
This widget takes an ImageURL, a web hosted location of the blank for, feel free to use “https://i.ibb.co/XLp9RrW/kaizen-audit.jpg” as a working example, but this could be any publicly accessible image file.
It also takes a font size (self explanatory) and a list of text, x and y positions. When the save() function is called the canvas is encoded as base64 and passed back as an event. From here it can be stored to a table.
Fair Warning - This widget really was a proof of concept for me, its far from perfect, and hasnt gone through Tulip’s standard code review process. Use it at your own risk. customWidget-Form V2.json (1.9 KB)
I have also used custom widgets with prop text inputs that were base64 to then pass them to API endpoints for stuff like anomoly detection. I can’t share that full widget because it includes some API keys, but here is the meaningful JS:
getValue('imageBase64String', () => {detectAnomalies();});
function b64toblob(image) {
const byteCharacters = window.atob(image);
const byteNumbers = new Array(byteCharacters.length);
for (let i = 0; i < byteCharacters.length; i++) {
byteNumbers[i] = byteCharacters.charCodeAt(i);
}
const byteArray = new Uint8Array(byteNumbers);
const blob = new Blob([byteArray], {type: 'application/octet-stream'});
return blob;
}
function getPred(preds){
let pre = preds[0];
console.log(pre.probability)
if(pre.probability > .7){
fireEvent('Color',pre.tagName)
}
else{
fireEvent('Color','No Match ' + pre.probability)
}
}
async function detectAnomalies() {
let image= getValue("imageBase64String");
const url = 'https://eastus.api.cognitive.microsoft.com/customvision/.....';
const img = b64toblob(image);
$.ajax({
url: url,
type: 'post',
data: img,
cache:false,
processData: false,
headers: {
'Prediction-Key': 'e6abf....',
'Content-Type': 'application/octet-stream'
},
success: (response)=>{
console.log(response["predictions"]);
getPred( response["predictions"]);
},
error: (err)=>{
console.log(err);
},
async: false,
});
}
After that, I tried decoding Base64 strings to a PNG image using a web site. I found that the Base64 strings that my custom widget output was too short from the original image’s Base64 strings. If I use some images which are stored in other sites, that worked well like Pete-san did in the previous post.
I would like to upload an image using an Image widget and convert it to Base64 strings then put it to “Azure Form Recognizer” to get strings from a form’s image via Rest API. Please check it out and give some advices.
Vision widget already have this function. So, only Windows PC can can convert an image to Base64 strings. Some Web APIs like Google Vision or Azure Form Recognizer can accept Base64 strings to receive images for OCR.
I think if a custom widget covers it, it’s OK. Image widget should be simple for everyone.
Totally agree that we need to unify the experience of using Tulip Vision and the normal image input widget. There was a feature request for this floating around, but that ticket had stalled.
Let me connect with the developers today to see if we can’t sneak this in as a quick-win
Let me add something. I guess “Image URL” contains data URI which starts from like “data:image/png;base64, iVBORw0KGgo…”.
Azure Form Recognizer needs us to send the only PNG data without the header such as “data:image/png;base64,”. I want to extract the part of PNG data only.
customWidget-File Encoder - PNG.json (2.3 KB)
Hi @emendoza , I’m not sure this is what you requested. You can upload a png file and this widget will convert it to base64. I hope it helps.
Hi @Akira the widget you shared me, needs a file upload from Windows, I’m looking for a widget to take data (image type) directly from variables/table records and then convert to base64, is that possible?
@emendoza
Well, if you want to get base64, please use Vision camera widget. It can convert image to base64 directly. I don’t think custom widget can do the same thing.
