Distributed Systems Pattern: Lease

Cluster nodes need exclusive access to certain resources. But nodes can crash; they can be temporarily disconnected or experiencing a process pause. Under these error scenarios, they should not keep the access to a resource indefinitely. more…

4d

The Lies that can Undermine Democracy

Like many Americans, I was transfixed and horrified by the recent assault on the Capitol. Much of this anger originates in lies perpetrated by irresponsible politicians and spread through media agencies. Lies like this can destroy democracies, and while we must have free speech we must not be free of the consequences of that...

5d

Maximizing Developer Effectiveness: Feedback Loops

Tim continues his comparison of high and low effectiveness organizations by comparing their key feedback loops. To improve these, organizations need to understand the importance of micro feedback loops, which are often neglected because they are so small. more…

2w

Distributed Systems Pattern: Consistent Core

Unmesh has a few more of his Patterns of Distributed Systems ready to share with the world. In this one he looks at the problem of a large cluster, one that is too large to effectively maintain strong consistency, yet needs to maintain some data in a strongly consistent way. It can do this by using a smaller cluster, which...

2w

Maximizing Developer Effectiveness

My colleague Tim Cochran has helped many software engineering organizations transform to respond faster to changing market needs. Often companies struggle with these transformations and a primary reason for these problems is that engineering organization has neglected to provide developers with an effective working environment....

2w

My favorite musical discoveries of 2020

Like most people, I'm looking forward to seeing 2020 in the rear-view mirror, but even this ugly year has brought some good things. For the last three decades I've regularly bought a few albums every month, and I thought I'd pick out a half-dozen favorites in the hope that they lead some readers to share at least a bit of...

4w

Data Mesh Principles and Logical Architecture

Last year, my colleague Zhamak Dehghani introduced the notion of the Data Mesh, shifting from the notion of a centralized data lake to a distributed vision of data. Based on more thinking, and the lessons of a year's worth working with clients, she's now written an article outlining four foundational principles of a data...

Dec 2020

Don't put data science notebooks into production

We've come across many clients who are interested in taking the computational notebooks developed by their data scientists, and putting them directly into the codebase of production applications. My colleague David Johnston points out that while data science ideas do need to move out of notebooks and into production, trying...

Nov 2020

Bliki: ComputationalNotebook

A computational notebook is an environment for writing a prose document that allows the author to embed code which can be easily executed with the results also incorporated into the document. It's a platform particularly well-suited for data science work. Such environments include Jupyter Notebook, R Markdown, Mathematica, and Emacs's org-mode....

Nov 2020

photostream 125

Melrose, MA (2020)

Nov 2020