Roman Podoliaka's blog

Counting new lines faster, or how to beat wc at its own game

Sun, 12 Oct 2025 00:00:00 +0000

Introduction Link to heading

If you haven’t heard of The One Billion Row Challenge, it’s a fun contest in which participants are asked to write a computer program that would parse a one billion line long text file and aggregate that data in the fastest way possible. This, of course, has very little to do with real life coding, but, nevertheless, it is a highly entertaining exercise in performance optimization.

debuginfod: easier debugging on Linux

Mon, 19 May 2025 00:00:00 +0000

What happened Link to heading

Since 2022, popular Linux distros (Arch Linux, Fedora, Ubuntu, Debian) have been serving DWARF¹ debug symbols and the source code for the packages in their repositories using debuginfod.

debuginfod is a web service that distributes ELF binaries, DWARF debug symbols, and source code over HTTP. Debuggers, profilers, and other tools can now lazily fetch that information from a remote server to help you analyze program crashes, reliably display stack traces, and so on.

The perils of temporary object lifetimes in C++

Tue, 21 Nov 2023 00:00:00 +0000

The other day my junior colleague asked me to help him troubleshoot a failing unit test: the actual value of a protocol buffer string field did not match¹ the expected value, and he couldn’t figure out why. I’m not allowed to share that code, but it can be summarized using this contrived example:

#include <iostream>
#include <string>

std::string foo() {
 return "Hello, programmer!";
}

int main() {
 std::string_view bar = {foo()};
 std::cout << bar << std::endl;

 return 0;
}

A seasoned C++ programmer will immediately spot the problem, but someone who have only used a memory-safe language like Java or Python before will be baffled why a seemingly trivial program above may or may not print the string "Hello, programmer!\n" depending on the chosen level of optimizations, the version of a compiler, operating system, etc.

Debugging CPython processes with gdb

Sun, 10 Apr 2016 00:00:00 +0000

pdb has been, is and probably always will be the bread and butter of Python programmers when they need to find the root cause of a problem in their applications — it’s a built-in and easy to use debugger. But there are cases when pdb can’t help you, e.g. if your app is stuck somewhere, and you want to attach to a running process to find out why, without restarting it. This is where gdb shines.

The story of one subtle optimization

Sun, 24 Feb 2013 00:00:00 +0000

In my previous job I worked on a web-service that was passing JSON payloads between workers using the Gearman task queue. One day I was trying to figure out why it took such a long time (a few seconds) to transfer a 3-megabyte JSON value. I was sure that it was a problem in our code. To my surprise, the profiler showed that the bottleneck was in this function of python-gearman client library:

Iterators for Beginners. The Way of С++

Sun, 20 Jan 2013 00:00:00 +0000

This post is mainly for beginners and is not intended to teach you everything about iterators in C++. So if you know about iterator_traits<> or advanced concepts like that, you will not learn anything new. Anyway, I hope this information will be useful for people who haven’t used C++ iterators before.

The problem Link to heading

Lets consider the following example: we need a function that would return an index of the maximum element in a given array of integers. We could implement it like this:

Mon, 01 Jan 0001 00:00:00 +0000

About me Link to heading

Hey! I’m Roman, a software engineer from Ukraine who currently lives in Ireland.

My programming language of choice is Python, as it allows me to be efficient and easily express my thoughts. I used to do some C / C++ / Java programming in the past and keep using those languages, but usually only when I need to read or modify someone else’s code. I’m also a big fan of Rust and Clojure and use them occasionally for various pet projects.

Mon, 01 Jan 0001 00:00:00 +0000

Talks Link to heading

2018 Link to heading

Continuous Integration: slides, video

2017 Link to heading

Machine Learning 101 at KharkivPy #17: video, slides, code

Debugging CPython processes with gdb at KharkivPy #15: video, slides

2016 Link to heading

Switching to the new EngineFacade: slides

Debugging CPython processes with gdb at PyCon PL 2016: slides, video

Python automated testing 101: slides

CPU profiling of Python applications at PyCon UA 2016: video, slides

CPU profiling of Python applications at KharkivPy #12 (Kharkiv, Ukraine): video, slides

2015 Link to heading

Debugging (C)Python applications at KharkivPy #11 (Kharkiv, Ukraine): video, slides

Mon, 01 Jan 0001 00:00:00 +0000

Switching to the new EngineFacade Link to heading

by Roman Podoliaka

@rpodoliaka irc: rpodolyaka rpodolyaka@mirantis.com

slides: http://podoliaka.org/talks/

The new EngineFacade Link to heading

spec: https://specs.openstack.org/openstack/oslo-specs/specs/kilo/make-enginefacade-a-facade.html
implemented by zzzeek (Mike Bayer) in Liberty (oslo.db >= 1.12.0)
a new clean API for using of oslo.db
- thread-safe initialization
- declarative management of session / connection scope
- easier offloading of read-only transactions to asynchronous replicas

“old” EngineFacade: thread-safe initialization is left up to users Link to heading

_ENGINE_FACADE = None
_LOCK = threading.Lock()

def _create_facade_lazily():
 global _LOCK, _ENGINE_FACADE
 if _ENGINE_FACADE is None:
 with _LOCK:
 if _ENGINE_FACADE is None:
 _ENGINE_FACADE = db_session.EngineFacade.from_config(CONF)
 return _ENGINE_FACADE

“new” EngineFacade: lazy thread-safe initialization Link to heading

from oslo.db import enginefacade as sql

@sql.reader
def some_api_method(context):
 # work with context.session

@sql.writer
def some_other_api_method(context):
 # work with context.session

“new” EngineFacade: multiple instances of EngineFacade Link to heading

main_context_manager = sql.transaction_context()
api_context_manager = sql.transaction_context()

“new” EngineFacade: project specific configuration Link to heading

context_manager = enginefacade.transaction_context()

context_manager.configure(sqlite_fk=True)
context_manager.append_on_engine_create(set_hook)

“old” EngineFacade: get_session() boilerplate Link to heading

@require_context
@handle_db_data_error
def snapshot_create(context, values):
 values['snapshot_metadata'] = \
 	_metadata_refs(values.get('metadata'),
 models.SnapshotMetadata)
 if not values.get('id'):
 values['id'] = str(uuid.uuid4())

 session = get_session()
 with session.begin():
 snapshot_ref = models.Snapshot()
 snapshot_ref.update(values)
 session.add(snapshot_ref)

 return _snapshot_get(context, values['id'],
 session=session)

“old” EngineFacade: private DB API functions Link to heading

@require_context
def _snapshot_metadata_get(context, snapshot_id,
 session=None):
 rows = _snapshot_metadata_get_query(
 	context,snapshot_id, session).all()
 result = {}
 for row in rows:
 result[row['key']] = row['value']

 return result

“new” EngineFacade: session scope is managed declaratively Link to heading

@main_context_manager.writer
def quota_destroy_all_by_project_and_user(context,
 project_id,
 user_id):
 model_query(context, models.ProjectUserQuota,
 read_deleted="no").\
 filter_by(project_id=project_id).\
 filter_by(user_id=user_id).\
 soft_delete(synchronize_session=False)

What projects are already using the new EngineFacade? Link to heading

Need help? Link to heading

Slides: http://podoliaka.org/talks/

Mon, 01 Jan 0001 00:00:00 +0000

Debugging CPython processes with gdb Link to heading

PyCon PL October 15th, 2016 Ossa, Poland

by Roman Podoliaka, Development Manager at Mirantis

Goals of this talk Link to heading

make gdb “a known unknown”, so that you consider it as an option in the future
highlight the common gotchas

Why debugging? Link to heading

working on a huge open source cloud platform - OpenStack