The Tale of Creating a Distributed Web Crawler

#229 – September 17, 2017

this week's favorite

The Tale of Creating a Distributed Web Crawler

This is the full story of how I ended up creating my own large-scale, distributed web crawler (in Python!). And especially how I dealt with the various technical challenges that ensued.

How JavaScript works: memory management + how to handle 4 common memory leaks

A few weeks ago we started a series aimed at digging deeper into JavaScript and how it actually works: we thought that by knowing the building blocks of JavaScript and how they come to play together you’ll be able to write better code and apps.

Face ID, Touch ID, No ID, PINs and Pragmatic Security

I was wondering recently after poring through yet another data breach how many people actually use multi-step verification. I mean here we have a construct where even if the attacker has the victim's credentials, they're rendered useless once challenged for the authenticator code or SMS which is subsequently set.

Breaking Up the Behemoth

I've been thinking about how applications evolve, and what we might do if we're unhappy with the results. Three apparently unrelated ideas have been percolating in my head. In this newsletter I'll introduce each one and connect them together, in hopes that understanding these connections will help us understand our apps.

So what’s this GraphQL thing I keep hearing about?

The trick to maintaining your sanity in these fast-moving times is to learn new stuff right between stages two and three, once your interest is piqued but while you’re still ahead of the curve.

newsletters