Developer Jess

This is going to be just a quick entry today, and when I was thinking of what to write about, it dawned on me just how much research I had gone in to in making different types of requests (https) in Node.  Took me a while to first have it so it was able to switch between a GET and a POST of JSON data, but then when I needed to make changes on how I would post Form data (including a file attachment) was another kettle of fish - steam rolled at every turn. So - let me help take you out of your misery.  Here is my base function and its required required libraries: An example of using this would go like follows. form = {      file: '@/tmp/path/to/filename.txt',      comment: '' }; request("POST","www.server.com","/",{},{'Custom-Header':'Custom-Value'},form) .then(res=>{ //console.log(res); //... }) .catch(err=>{console.log(`Request Error: ${err}`)}); This will send form data, including the filename you specified in to the ...

I'm genuinely excited to write this particular entry, for two reasons.  One is that its been some time since I've jumped back in to NodeJS, but the other because is because it is my first attempt at writing something using Puppeteer .  I needed to scrape information about products on a particular website, and through my exploration, on each iteration of the page parameter in the query string, in the Chrome Developer Tools , there was an API request for the granular information that I needed for the products. It was extremely difficult to work out just how this URL was being constructed in order to know what to request for, but I quickly came to a conclusion that if there was information in the developer tools, then that could be somehow extracted - and remembered this little tool existed - so I hopped to work! So the premise came across as being rather simple - intercept the browser making the requests that you find in the network tab .  There was one path that I knew...