Geo-spatial in-memory caching using Rtree

Adam Lev-Libfeld

A long distance runner, a software architect, an HPC nerd (order may change).

Latest posts by Adam Lev-Libfeld (see all)

Having to save a lot of geo-spatial data is a task rarely one passes without being scared for one’s life. Whether you are using a DB like postGIS, an in-memory data store like geoRedis or some other tool, handling geo-spatial data usually mean using RTree. Using RTree directly can ave some very positive effects on your system performance, so we thought we’ll let you in on this industry secret. Continue reading “Geo-spatial in-memory caching using Rtree”

Geo-spatial in-memory caching using Rtree

better event-driven programming using flexible state

Adam Lev-Libfeld

A long distance runner, a software architect, an HPC nerd (order may change).

Latest posts by Adam Lev-Libfeld (see all)

This is a true relic- the few who can spend the time and read it will be rewarded with the gift of understanding state. Using the complex example of…. a calculator:

At first glance, this approach seems to work just fine. Indeed, when you launch the calculator (available for download at <www.cuj.com/code>), you will certainly find out that most of the time it correctly adds, subtracts, multiplies, and divides. What’s there not to like? However, play with the application for a while longer, and you’ll discover many corner cases in which the calculator provides misleading results, freezes, or crashes altogether.

Source: Who Moved My State? | Dr Dobb’s

Link

3 Tips To Not Holding Back Your Geo-Data

Adam Lev-Libfeld

A long distance runner, a software architect, an HPC nerd (order may change).

Latest posts by Adam Lev-Libfeld (see all)

It’s been almost a decade that I’m in the HPC (≈cloud≈scalable software) field. During these joyful years, the family of problems which seems to pop back the most are those how are heavily dependent on geographic data as the main input (not that there’s anything wrong with that).

One would expect by now that these kind of problems will be solved by now, but the main source of (relatively) reliable GPS data today, in terms of volume, are smartphones, and these weren’t really here a decade ago.

Continue reading “3 Tips To Not Holding Back Your Geo-Data”

3 Tips To Not Holding Back Your Geo-Data

Andrew Montalenti – streamparse: real-time streams with Python and Apache Storm – PyCon 2015 – YouTube

Adam Lev-Libfeld

A long distance runner, a software architect, an HPC nerd (order may change).

Latest posts by Adam Lev-Libfeld (see all)

A nice introduction to one the backbone tools we use here at Tamar. Streamparse allows us to rapidly transform regular python code to a  apache storm runable  topology.

Link

Breaking the monolith – How to design your system for both flexibility and scale – Part 2: The Cathedral & The Bazaar

Adam Lev-Libfeld

A long distance runner, a software architect, an HPC nerd (order may change).

Latest posts by Adam Lev-Libfeld (see all)

This post is part of a series. you’d probably like to read it from the beginning. You may also be interested in the next part of the series.

 

As a paraphrase on Eric Raymond’s famous book under the same title (O’reilly ,open format) let us, for a moment, think of the church as a computing system. The  service (no pun intended) is led by the priest, and has a single, general, output, it is very reliable (when comparing to other medieval services) but given under very strict terms (start time, end time, location, donation). If you don’t like your town’s priest, the next best option (if you are a medieval peasant)  probably involved some serious walking and certainly was not the safest things you could do in your day off (how’s that for a captive audience?) .

The Bazaar, On the other hand is a whole different animal Continue reading “Breaking the monolith – How to design your system for both flexibility and scale – Part 2: The Cathedral & The Bazaar”

Breaking the monolith – How to design your system for both flexibility and scale – Part 2: The Cathedral & The Bazaar

How to Do Performance Calculations the Right Way

Adam Lev-Libfeld

A long distance runner, a software architect, an HPC nerd (order may change).

Latest posts by Adam Lev-Libfeld (see all)

Knowing what are the computing capacity of your cluster (on-site or not) is a critical part of designing a made-to-last software. Getting your math right might just make the difference  between a great launch and an PR catastrophe

the math

The first thing we need to know about performance calculations is that it is a mathematical tool to help us estimate what our machines, servers, CPUs, or lab-rat-brain-computational-array are capable of.

Basically what we want to do is to figure all the different elements this formula out:

Continue reading “How to Do Performance Calculations the Right Way”

How to Do Performance Calculations the Right Way