Parse HTML from URLLoad through {site}

[Peter_Smulders]

The {site} command can already parse (complete) HTML pages. The {urlload} command can pull in complete HTML pages, if you use the body response.

Please either clone the site command or extend it to accept an optional parameter to parse not the current page, but whatever {urlload} produces in the body response, so that that response can be accessed through CSS selectors. (currently, we are limited to regex parsing)

I understand the {site} needs a valid and complete page to parse the DOM, but the urlload command provides that just as much as the current page body does.

[Gaurang_Tandon]

Hi @Peter_Smulders can you please also describe your specific use case - why it would be helpful for you to have this functionality? Note that {site} can already pull in information from other open tabs in the browser:

November Developer Update

Text Blaze Updates Integrating Site Data Across Multiple Chrome Tabs You might already be familiar with the {site} command, which allows you to pull content from a website and use it in your snippets. Up until now, this command was only able to get data from the tab you're currently on. But guess what? We're stepping up our game with a new Beta feature! Now, you can extract data from any Chrome tab that is open at the time of inserting the snippet! In the animation …

Let me know if that's what you were looking for.

[Peter_Smulders]

Gaurang and myself have discussed this away from the forum, but for community's sake here is the summary:

My use case for this feature is that the pages I need to pull information from need to be reloaded to get current information. (pages are static; underlying data is not) {urlload} does that, where {site} does not. Sure, I can open the page, trigger the snippet and then close it again, but that seems far clunkier than it should be.

So, ideally, I would have {urlload} hit a particular page (with a URL than I can craft with some parameters) and use CSS selectors to parse the output. CSS selector parsing like {site} does works well, plays nice with storing series of values in lists, etc. The alternative is setting up regexes to do the same thing. As powerful as regexes are, that is much, much uglier and more difficult than just going table > tbody > tr > td:nth-child(4), etc.

Anyway, even though {site} and {click} have this capability to parse CCS selectors, which makes it somewhat feasible to hook this up to {urlload}, it still is actual development work, which will not readily get a lot of priority because the subset of users that will actually find use for this is very small. (and development time should go to better serving the largest amount of customers)

(Note: if you are in that small community and would like to show that you actually would find this useful, post a comment with your use case and/or vote. We can always hope. :))