From time to time, my usual debugging tools: good ol printf, logger.log, console.log, gdb, pdb are not enough to solve the problem. This happens more than often when multiple processes/threads are involved or performance is an issue.
One such tool that I found really useful is strace.
Linux Performance Observability Tools. Source: https://www.brendangregg.com/Perf/linux_observability_tools.png See man strace (great read)!.
TL;DR
Traces all syscalls (aka how your program interacts with os) It’s like printf but it automatically prints everything being read/written to any file descriptor (file, socket etc.
Python’s multiprocessing.shared_memory module provides a way to share a block of memory between processes. It’s usefull in ml contexts, where an app will runs ML model in separate process and wants share the results with its parent process. Frigate NVR does exacly that, see: object_detection.py.
# Define properties of shared memory SHARED_MEMORY_NAME = "shm-1" SHARED_MEMORY_SHAPE = (5, 5) NP_DATA_TYPE = np.float64 # Calculate shared numpy array size dsize = np.dtype(NP_DATA_TYPE).itemsize * np.
The error HangupException The remote server unexpectedly closed the connection. at /usr/local/lib/python3.10/site-packages/dulwich/protocol.py:215 in read_pkt_line 211│ 212│ try: 213│ sizestr = read(4) 214│ if not sizestr: → 215│ raise HangupException 216│ size = int(sizestr, 16) 217│ if size == 0 or size == 1: # flush-pkt or delim-pkt 218│ if self.report_activity: 219│ self.report_activity(4, "read") The following error occurred when trying to handle this error: HangupException Host key verification failed. at /usr/local/lib/python3.10/site-packages/dulwich/client.py:1154 in fetch_pack 1150│ with proto: 1151│ try: 1152│ refs, server_capabilities = read_pkt_refs(proto.
It’s time to talk about Pythons Literals and I mean that literally :smile:.
Now that we got that unfunny joke out of the way.
What are Literals and why are they usefull The basic motivation behind them is that functions can have arguments that can only take a specific set of values, and those functions return values/types change based on that input. Common examples are (you can find more here):
Basic CORS settings for Django Rest Framework (DRF) + Vuejs:
# Settings.py: ... CORS_ORIGIN_ALLOW_ALL = False CORS_ALLOWED_ORIGINS = [ "http://localhost:5173", ] CSRF_TRUSTED_ORIGINS = [ "http://localhost:5173", ] CSRF_COOKIE_SECURE = True SESSION_COOKIE_SECURE = True CSRF_COOKIE_SAMESITE = "None" SESSION_COOKIE_SAMESITE = "None" CORS_ALLOW_CREDENTIALS = True CORS_ALLOW_HEADERS = [ "accept", "authorization", "content-type", "user-agent", "x-csrftoken", "x-requested-with", "x-xsrf-token"] ... In Vuejs:
// axios config: xsrfCookieName: 'csrftoken', xsrfHeaderName: 'X-CSRFToken', withCredentials: true, Read what these do before using them. I’m not responsible for any security issues that may arise from using these settings.
Poco F1 - Restarts on Fastboot (AMD) I tried to flash custom recoery for my Poco F1 (codename “beryllium”). I installed ADB drivers and drivers for my phone. Windows device manager said everything was ok so I procedded and entered fastboot mode (Volume down + Power). I entered command:
fastboot devices And the fastboot logo disappered and was replaced with text in top left corner: “Press any key to shutdown”. Then I remembered that the guide said this:
TL;DR Some apps work flawlessly and some (those with postgres) take a lot of time to initialize.
I don’t write this post to rant about my bad experience, it was completely my fault. But to issue a warning for a newbies, to save them from my pain (and hours of debugging).
Btw: the Truecharts team is awesome, they provide a ton of apps and their support is top notch. Please consider supporting them on Patreon.
In my previous article, I showed how to create a simple search engine using OpenSearch and its fuzzy query. This time I’ll show something much more powerful - semantic search engine.
Getting started To build our search engine, we’ll need:
Embeddings of text. Embeddings enable algorithms to do similarity search, a search that can find sentences with words not used in a query, that had a similar meaning to those in a query.
In this tutorial, I’ll show you how to generate embeddings for sequences in polish using Sequence Transformers. I won’t explain how they work, there are many great articles:
Measuring Text Similarity Using BERT Bert 101 What we’ll need is a Sequence Transformers library from Huggingface:
pip install sequence_transformers The code is simple, we import library, create model and ask it for embeddings.
from sentence_transformers import SentenceTransformer model = SentenceTransformer('Voicelab/sbert-base-cased-pl') embeddings = model.
Imagine that you’re working with a large text dataset. It can be anything:
a set of tweets scraped articles You have a client that wants to find some information in this dataset or do some analysis with it. You probably cannot fit it inside a single Pandas data frame and query it. That’s where tools like OpenSearch and OpenSearch Dashboards can be useful. You can write a query for a specific term, set of terms or even do fuzzy matching.