Project: KeyWordExtraction from CSV
Using PapaParse, I’m able to parse out a relatively large CSV file quickly into a JS Array..of Objects. My biggest issue with the way PapaParse returns the data is that it doesn’t allow me to choose the first field as the object key, so I’ve been learning the .filter function (though I was worried that iterating over an array of objects in JS would be slow especially with a dataset that large). The takeaway here is that I can run a custom function on each ELEMENT of the array (and pass the anonymous function a temporary reference named however I like). In this case my function looks like this:
var newArray = workingResArray.filter(function(el){ // el is a reference to each array ELEMENT that the filter is iterating through for (var i=0; i<searchArray.length; i++) { if (el['Long Name']==searchArray[i]){console.log("we found:",searchArray[i]);return el;} } }); // end of anonymous function inside of filter
This wasn’t immediately apparent for me and took a little bit of tinkering (It’s been a while!). Inside these functions I searched for the relevant key data and in my latest iteration I load all the relevant data from the export (now 200k rows and about 18mb). A search of 20 unique entries takes about 80ms total to compile the relevant data and write it out to a CSV for one of our plugins, cool!
Leave a Reply