Fast Learner Tip: A Visual Introduction to Machine Learning

Adam Lev-Libfeld

A long distance runner, a software architect, an HPC nerd (order may change).

Latest posts by Adam Lev-Libfeld (see all)

What is machine learning? See how it works with our animated data visualization.

Source: A Visual Introduction to Machine Learning

 

When approaching a performance problem this method of train and test is, sadly, not yet fully applicable. This is why we have to keep our software as scalable as possible, to enable it to fit on the go with increasing (or falling) demand.

The ability to train on real time data (or learn from mistakes) using a closed feedback loop is a common method to do adaptive machine learning, and is exactly the same as we use to determine how many instances of our software currently needs to be up.

In Storm, circles in topologies are usually looked upon as a perversion, but with a couple of simple tricks you can always move the model out of Storm to a datastore (Redis maybe?), and separate the training bolt from the predicting (or classifying, or testing, depending on what stage you are) bolt.

Link

Understanding State in a Stateless World

Adam Lev-Libfeld

A long distance runner, a software architect, an HPC nerd (order may change).

Latest posts by Adam Lev-Libfeld (see all)

It has come to my knowledge, that in the last post in the “breaking the monolith” series failed to clarify the shift needed when moving from a statefull (usually object-oriented) design to a stateless (usually event-driven) system.

As you probably already know very well, as workers in the information industry we dabble in the fickle trade of moving, storing, and manipulating information (either into other types of information or into actions). We also like to call it “data”, so we would feel smarter (and cool) and other people would think that what we do is extremely complicated and hard. Let me assure you, it is neither.

Continue reading “Understanding State in a Stateless World”

Understanding State in a Stateless World

Breaking the monolith – How to design your system for both flexibility and scale – Part 4: The Message

Adam Lev-Libfeld

A long distance runner, a software architect, an HPC nerd (order may change).

Latest posts by Adam Lev-Libfeld (see all)

This post is part of a series. you’d probably like to read it from the beginning. Or check out the previous post in the series

Those of us who had the chance to experience the exquisite pleasure of getting a formal education in the field of software engineering were told a fabulous tale. This tale of extreme bravery and audacity  included mythical creatures and ideas – like a zero-downtime zero-loss infrastructure, instant globally synchronizing data (allowing the creation of what is sometimes referred to as the “single point of truth”),  and of course – the object oriented system.

Continue reading “Breaking the monolith – How to design your system for both flexibility and scale – Part 4: The Message”

Breaking the monolith – How to design your system for both flexibility and scale – Part 4: The Message