Overview

There isn't much to "see" when it comes to data engineering and backend web development unless you're really into poring over logs. Similarly, reading arcane code snippets with no explanation is less than exciting. To help demonstrate my expertise, these projects exist with the blog to help "show" what I can do.

This project is what you're looking at right now. I built it because most of my work has been in databases and data analysis, but I wanted to be able to show some diversity in my skillset. The site is hosted on Amazon Lightsail Cloud Server running Ubuntu. Ubuntu is running MariaDB, Nginx, Gunicorn, and Django (Python). Frontend development is clearly not one of my skills.

This project helps students of Spanish choose books appropriate for their reading level. It creates graphs describing the distribution of words over different Cervantes levels (A1, A2, etc.). It also gives information about all of the words in the book including how often the word is used in the book, how common it is in Spanish overall, and its definition(s). With this information, users can select the most important words and download a deck of Anki flashcards to help them memorize these words before even starting the book.

This project allows the user to see a network of Twitter users' follower and followee relationships.

• Network Visualization
  The user enters two Twitter usernames. The server connects to a database of Twitter users and returns user and connection data to create a network graph of users. The layout of the network shows not just the follower and followee relationships, but also how they compare between the two Twitter usernames.
• Twitter User Request Queue
  Twitter is pretty big (currently ~300 million users) and its rate limiting doesn't allow on-demand querying. If a user wants to see Twitter users that don't exist in the database, the user can add the usernames to the Request Queue and the user will be fetched in the order submitted.
• Automatic Database Growth
  Every minute, information for a Twitter user from the Request Queue is retrieved from Twitter. If no users exist in the queue, a user is selected at random.