Skip to main content

Posts

Liberating Light Data - Part 2

After much research, I felt it was only fair to share the knowledge, and so picking up from one of my last posts where I demonstrated how to get data, now it's time to dive in to how to write data to Google Sheets.  The rest of the class can stay the same, we're just going to add one function. So what's happening here? The main thing to point out is that we want to pass an array of arrays for the values, representing the rows and columns.  If you are going to be wanting to do that in an API, might not be as simple without serializing and base64 encoding for good measure. Otherwise, it's fairly self-explanetory.  I'd be happy to try and answer any questions, but hope it inspires!

Back in to Dashboard Reports

Carrying on from my last post , a while back, I blogged about how I took data from Campaign Monitor and posted to Google's Data Studio for the sake of creating a better, compiled dashboard.  Well the request popped up again for me, but this time to take some more generic data, so I decided to write a better class for it. This is going to be short and sweet, like the previous post, we're going to need a simple addition to our composer.json: The construct object is going to look very similar as well: And now we'll just make a simple upload function to use to store in Google's storage buckets: From there - you can go nuts in the Data Studio.  Folder and file naming conventions are very important, with spreadsheets for instance, you can keep uploading date stamped files which will merge nicely and you can see your increments over time. Hope you'll all be able to make some cool integrations with this! J.

Liberating Light Data

On to a new topic today, we've all heard the buzz about "Big Data" and how we can tackle querying in lightning fast speeds, but what are some of the best ways of working with highly mutable but tiny data sets? I'll lead you in by example on this one - I had recently had a custom developed PIM (Product Information Management) system, but also part of this system, I also kept a database of our stores and their respective catalogue versions. For a while, it was good - until it wasn't. It didn't scale that effectively in the long term. The PIM should have only been just that, not a hybrid of extraneous bits and pieces of data. So I'm going to fast forward to the solution its now been moved to - I've created a workbook in Google Sheets and tied the integrations to this. This not only scales incredibly well, but you also have the luxury of cell-by-cell history out of the box. So, how to proceed? First you need to add the component using Composer : ...

Going Serverless!

So most of you have probably heard about this particular concept - " Serverless ".  I can't say I completely get the name of it, there is a server that is running it, the main difference is it only runs for the length of it to take to complete the computing of the function.  As far as the big players go - I generally have all of my stuff running out of Amazon - so for the purpose of this article, this is what I'm referring to. Lambda is the service of AWS where you can run your serverless functions.  Now sadly for us PHP developers, it does not support natively support PHP and until now, I haven't found a simple and easy to grasp guide without the need to use some elaborate framework that sits on top of it.  Until now... Be sure to take a look in to this guide from Amazon.  I was genuinely impressed.  The one thing it left out was that I got about up to the very end when I found my functions could not be invoked properly, and the reason being is that m...

WooCommerce: Turbo Product Imports

Well, this is really becoming a topic as part of a series !  Today's topic is going to be a little complicated, but it can really speed up any current integration's you may have between your WooCommerce and CMS/Product Management Systems you may have in place.  Let me first explain my current situation. The main issue, which turns in to a cumbersome work-around, is that our imges are all hosted on an external CDN.  As I have already covered , by using the REST API, despite you give the URL for each product's image, it still uploads it to the media library.  This comes at a cost of time, and for a heavy workload can be prone to connection timeouts. As part of my role, I put through many catalogues to a site, with a combined number of products in the 1000's - and it was quickly having to scale, so there had to be a better way.  I knew there were people who had desired to write posts directly to SQL , and what is a WooCommerce product? A custom post! Think L...

Options outside of ML

Today I wanted to share what I have done in a recent experiment to address an issue that has been quite rapidly getting out of control.  We have a responsibility of pushing out a number of catalogues on a week by week basis, and since the source information has got product labels truncated down due to the size of the original input field, we have got the manual task of renaming to their full displayed length. Of course, we have got our ways of reducing the amount of manual updates on this, but we certainly could not pick up on most things as fast as a machine could.  It is not as simple as just a "find and replace" though, because while such abbrieviations as "sce" mostly means "sauce, it's never a one-size-fits all, and depending on the brand, they can also each have their own spelling. I then came up with an idea; using some form of diff algorithm, and having a dictionary be learned associated with the products respective brand name.  The library I f...

Things I've Learned

Greetings, fellow devs! It's set to be a busy month, both professionally and with some well desired home projects as well.  I don't have anything too specific, but I wanted to touch on a couple of things I've learned lately and what I want to learn in the future in order to proceed. Python versus Golang I recently conducted an online poll , where I am looking to make a Google Cloud Function as a serverless API, which will be heavy in file processing.  I don't know either language half as good as I know PHP, so I am excited to experiment in new avenues.  As you can see, Golang was the clear winner. I like to learn by example, so I've found two main resources which I think will be what steers me in the right direction: Go By Example - in particular the section of reading/writing files, and Multithreading in Go Tutorial   Never accept standard error reporting   On one of the most important projects I am actively working on in my 9-5, for too long I ...