How to extract tables from variable URL
Is it possible to use the web recorder table extractor to get a table from a URL that contains variables? As far as I can tell, you need to know the exact URL string in advance.
I'm trying to automate the extraction of details related to open tenders from a website. All tenders are shown on a homepage in a stacked list of posts that contain high level information. The posts also contain a URL to a more detailed page. I'm trying to extract the detailed information by following those links.
My approach has been to use the table extractor from the web recorder to get the high level information from the main page, then to loop through the posts and use the ID column to run the second extraction. Ideally I could feed something like "tenders.com/?id=%tenderID%" into the URL field of the extract table command. Alternatively I could use the UI filter control to open the detailed page and use the extract table command on the active window.
I am unable to omit the URL field on the extract table command, which leads me to believe I can't pursue the active window approach, and the URL field will not accept variables.
Any guidance would be appreciated.