My experience: Designing data-intensive applications
It has been 10 days since the last blog post, and I am proud to say I have finished the book!
The journey was quite arduous, and I can confidently say that spending 40hrs reading a very technical book is not an easy task at all, though I enjoyed most of those 40hrs. The book gave me an in-depth understanding of various topics relating to both databases and distributed systems. I learned lots about how things worked in the background (like how files are stored), the db technologies available (relational vs document vs graph), problems with having a distributed system (read-write skews, linearizablility, isolation), as well as methods to deal and mitigate these issues. All in all, the book went across many topics, and due to the sheer volume of knowledge put in there, I cannot remember everything that I have read. Some topics where also quite confusing, notably: Modes of dataflow, secondary indexes, most of the technical aspects of stream processing (stream joins were especially confusing)
I would say that my understanding of this field has been greatly amplified, though definitely not enough to build a database from scratch by itself. Another observation is that while the theory was nice to know, I did not really get actual practice with any of these topics, and that would likely lead to me forgetting a decent portion of the book if I do not refresh my memories on these topics sporadically (I doubt this will be hard, seeing as distributed systems and database are some of the most relevant things in today’s web landscape).
It is for this reason that I am not yet comfortable undertaking my database project. In fact, I said in my previous blog post that I would be coding this up in GO. So, that was a lie 😂. I think I want to get really good at C++ instead, as it might open more opportunities for me in different fields of work (such as HFT as it requires low latency code). Instead, I will be undertaking another famous book: Crafting Interpreters, a book which will hopefully teach me more about interpreters and compilers. I head it is an amazing book with more practical work to consolidate my understanding. It was also recommended to me by a friend, so why not check it out!
Coursework season is coming soon, so I doubt I’ll be able to finish another book in time, but who knows what the future has in store for me. Until then, thanks for reading this post.