Implementing a TF-IDF (term frequency-inverse document frequency) index with Python in Spark

Introduction As part of the final exam assignment for my Masters in Data Science course “DS8003 – Management of Big Data Tools”, I created a Big Data TF-IDF index builder and query tool. The tool consists a script with functions to create a TF-IDF (term frequency-inverse document frequency) index and it is then used it […]

Analyzing Reddit Public Comments on Azure Data Lake and Azure Data Analytics (Part 1.5)

In the previous article on this series, I skipped the part where I downloaded data. At first I used my laptop and a downloader to get the files locally, which I ended up uploading to the Azure Data Lake Store folders. Another alternative that I wanted to give a try and will show you in […]

Analyzing Reddit Public Comments on Azure Data Lake and Azure Data Analytics (Part 1)

With some free time in my hands in between Coursera courses and classes not starting for the next couple of weeks, I wanted to use some of the new Azure Data Lake services and build a Big Data analytics proof of concept based on a large public dataset. So I decided to create these series […]

mongoDB – What’s great (and not so great) about it

mongoDB is a relatively new database management system, one of the prime examples of the No-SQL database movement (if such a thing exists). In No-SQL databases, that can also be referred to as ‘non-relational databases’, you don’t represent data tables that store rows and their relations. Each No-SQL database has its own particular way of […]