Accessing table data in custom widgets

Support, Troubleshooting, & Help
1

I want to create a custom table widget. The Widget API (at least what’s documented) only gives me access to getValue(), setValue(), and fireEvent(). I’m looking for a workaround.

From the widget code, you can do this:

fetch("/api/v3/w/1/tables/xxxxxx/records? ...",
    { credentials: "omit",
        headers: {
            "Authorization": "Basic jaslfjasfjaslfjkasljkgasgas=="        }
    })

but permissions are an issue. It would be nice to use credentials: "include" but that isn’t allowed because of CORS:

The value of the ‘Access-Control-Allow-Origin’ header in the response must not be the wildcard ‘*’

What I have done in the short term was hard-coded an API key, which isn’t great. I could pass the API key in as a widget parameter, but that isn’t a great solution, either.

My thought here is to propose a new feature, but I’m not exactly sure how to shape it. Some ideas:

  • Expose a tables API to the widget, in addition to the three methods listed above
  • Pass the API credentials into the widget or let it get them with a new exposed method, perhaps named getApiCredentials()
  • Add a new widget parameter type that is something like a RecordSet

These are just ideas, Tulip engineers probably have others.

I’m sure someone will ask so I’ll just volunteer up front: I’m trying to make a much fancier table widget.

[Update]

Even given the table ID, I still have to process the column names … using this:

const re = /^[a-zA-Z0-9]{5}_/

function origColumnNames(tbl) {
    tbl.forEach( row => {
        Object.keys(row).forEach( oldKey => {
            const newKey = oldKey.replace(re, "")
            delete Object.assign(row, {[newKey]: row[oldKey] } )[oldKey];
        })
    })
    return tbl;   
}
1 Like
2

Hey @wz2b,

All of these are great ideas and limitations we are working to address right now. v1 of custom widgets has pretty limited ability to do trigger actions intentionally. Adding functionality to be able to advance steps, open the menu, print step, etc is fairly easy, and they lack some of security/stability concerns that come with stuff like accessing table data directly. I would expect these functions to come before direct data access functions in custom widgets.

A little more context around those concerns:
Security – Custom widget code is running in an iframe in the player and it is running raw Javascript, so if we made a portal to allow you to directly access table data, that portal would also potentially be accessible from other javascript. The whole Bot architecture around protecting and monitoring access to your data would be obsoleted. The right approach instead would be to build out the sort of authentication process you are talking about on the widget level to access this data. Maybe in the widget UI you can tie it to a bot that you have already created and then you have access to functions to directly pull/edit your table data.

Stability – We do pretty extensive testing to insure tables can support the speed/scale at which Tulip apps can update tables, but adding javascript into the ring just makes the testing ask significantly more challenging. With Javascript you can fairly easily run loops faster than 50ms (far faster than you can loop right now in a trigger) and subsequently hit tables harder than triggers natively can.

There is a lot of thought right now going into being able to pass table queries as object lists into widgets, and your question around table field ids is one we are working through too, the prefix before table columns is a necessity to unsure field IDs are unique, but something we abstract away from users whenever possible.

Having said all of this—
Your approach hitting the api directly in your widget should enable direct table interaction/edits should enable all you need in the interim, and your feedback will be fed into the iteration on the feature.

This feedback is fantastic - Keep it coming!
Pete

3

Here is a somewhat disorganized collection of thoughts.

If there were a way to have custom libraries, I would want some way to interact with them that I could deploy from my IDE. This could be a new resource in the Tulip API.

I think these resources do need to be bundled somehow. Webpack was mentioned elsewhere; it’s not a bad idea. Yeah it makes the project a little trickier, but not much. What we could do is create a “Tulip Widget SDK” template that has a package.json with the right devDependencies and scripts definitions to build the bundle. For more casual developers, it would be a matter of download this sample project, type “npm install” then type “npm run build.” I don’t really think it’s that bad. I’m not up enough on this to know if instead of webpack it really should be some kind of WebComponent. One advantage of this approach is it would let me work in typescript, which I’d prefer (but that’s not a strong concern here in that generally speaking ‘widgets’ are smaller/simpler).

It’d be nice to be able to deal with these widgets as a complete bundle, with all the necessary html/css/js as one thing. It might be nice to be able to bundle multiple widgets into a single library. Node red approaches this by having a custom tag in package.json that declares the entry points for all of the nodes contained therein.

I would also like a way to run logic without a UI. The mechanism you have now almost works for this - you can fake it the way tracking cookies work on some internet sites, where you put a 1x1 invisible png on the page. But it’d be nice to just allow them without having to do that. What I think would need a little work here is that events are outbound (from the widget) only, and there is no clear lifecycle (that I can see) for sending events inbound. I think a good way to do that might be just through variables, my proposal would be a little extension to what you already have:

     getValue("someVariable", (newValue, oldValue) => {
         console.log("The value changed from", oldvalue, "to", newValue)
     }

On our other discussion, I still think I would figure out how to expose your API the same way you expose getValue() setValue() and fireEvent() … you could add a method getApi() that returns something I can make calls against that will handle the credentials for me.

4

These two things go together in my mind. I also keep thinking of SquareSpace “code blocks” where there’s a visual space delineating extended or custom functionality inside the WYSIWYG editor. See Code blocks – Squarespace Help Center

5

Hey @wz2b – I’m one of the engineers that built custom widgets. Thanks for the feedback! Thought I’d jump in directly to add a few things

We originally built a prototype of Custom Widgets that involved an external SDK + webpack to build a real bundle (and allow you to use Typescript, React, etc). While it was great for developers, it was also a much higher barrier of entry into Custom Widgets. That experience led us to the current approach of building a lite-IDE straight in the browser. Maybe we’ll add the SDK option later on, but planning to stick with the browser-based approach now

On adding direct API access from the widget – it’s definitely something I’d like to do eventually. As Pete mentioned above, the main concern is security. I’d like to add a permissions model. Think of what happens when you install a new app on your phone – you have to grant it permission to send notifications, use the camera, etc. Similarly, would like to have some permissions you grant to the widget when you install it. Custom widgets right now only have access to the data you pass in, so we haven’t needed this security model yet. So – coming eventually, but need to build more of a framework around it to maintain a strong security model.

6

Also, for parsing from column name to label – have you considered making a GET request to the /tables/{tableId} endpoint to pull up the mapping between name and label?

7

Yeah, I did that. Are you interested in seeing the whole thing? I’m experimenting here so it’s not really documented (commented) properly but I can at least give you an idea what I’m up to. Maybe it’ll give you some ideas.

const machine = getValue("machine").id;

const requestOpts = {
    credentials: "omit",
    headers: {
        "Authorization": "Basic REDACTED=="
    }
}

function findtable(tblName) {
    return fetch("/api/v3/w/1/tables", requestOpts)
        .then(response => response.json())
        .then(tables => tables.find(table => table.label == tblName))
}

function remapColumnNames(tbl) {
    const re = /^[a-zA-Z0-9]{5}_/
    return tbl.map( row => {
        Object.keys(row).forEach( oldKey => {
            const newKey = oldKey.replace(re, "")
            if ( newKey !== oldKey) {
                delete Object.assign(row, {[newKey]: row[oldKey] } )[oldKey];
            }
        })
        return row
    })
}


function convertTimestampColumnsToMoments(tblDescr, rows) {
    return rows.map( row => {
        Object.keys(row).forEach( colKey => {
            let colDescr = tblDescr.columns.find( colDescr => colDescr.name === colKey)
            if ( colDescr && colDescr.dataType.type === "timestamp") {
                row[colKey] = moment( row[colKey] )
            }
        })
        return row
    })
}

function getTableRecords(tblName) {
    return findtable(tblName)
        .then(tblDescr => fetch(`/api/v3/w/1/tables/${tblDescr.id}/records`, requestOpts)
            .then(response => response.json())
            .then(rows => convertTimestampColumnsToMoments(tblDescr, rows))
            .then(rows => remapColumnNames(rows))
        )
}

function updatetable(rows) {
    const config = getValue("config")
    const tbody = $("#pmTableBody")
    tbody.empty()
    rows.forEach( (row, i) => {
        let tr = $('<tr>');
        
        /*
         * Colorization rule - add the pastDue 
         */
        let c = ""
        if (row["due"] instanceof moment
            && row["due"].isValid()
            && row["due"].isBefore()
            && row["completed"] == false) {
                c = "pastDue";
        }
        
        console.log(row)
        
        config.forEach( cfg => {
            const content = row[cfg.columnName];
            if (content instanceof moment) {
                const formatted = content.isValid() ? content.format(cfg.format || null) : "--";
                $('<td>').text(formatted)
                    .click(  () => { onClick(row.id)})
                    .addClass(c)
                    .appendTo(tr);
            } else if ( typeof content == "boolean") {
                $('<td>').text(content ? "\u2705" : "\u274c")
                    .click(  () => { onClick(row.id)})
                    .addClass(c)
                    .appendTo(tr);
            } else {
                $('<td>').text(row[cfg.columnName])
                    .click(  () => { onClick(row.id)})
                    .addClass(c)
                    .appendTo(tr);
            }
        })
        tr.appendTo(tbody);
    })


    tbody.find('tr').forEach(  (row) => { console.log("row", row)})
    

}


function onClick(id) {
    fireEvent("Row Clicked", id)
}
function setupTable() {
    const headRow = $('#pmTableHeadRow')
    const config = getValue("config")
    if (config != null) {
        headRow.empty()
        config.forEach( cfg => {
                headRow.append($('<th>').text(cfg.heading) )
        })
    }
}

setupTable()

getTableRecords(getValue("tblName")).then( rows => updatetable(rows) )
8

I don’t know, I think you should have let me at the SDK and see what I could do with it before you decided that :slight_smile: I get though that you can’t do everything at once.

9

Nice work on the widget! Have you seen the “Export widget” button from the overview page? It’ll let you export the whole widget to share with someone else:

image
image3054×632 94.5 KB