Posts

Running a Local LLM on Mobile: Testing PocketPal on iPhone 12 With the increasing accessibility of large language models (LLMs), running them locally on mobile devices is an exciting prospect. I recently tested PocketPal, a mobile LLM interface, on my iPhone 12, using a distilled 4-bit quantized model. Here’s a breakdown of my experience, covering installation, performance, and overall usability. Why Run an LLM on Mobile? Running an LLM locally on a mobile device comes with several advantages: ...

Running DeepSeek-R1 1.5B on Raspberry Pi 5 (CPU-Only) Technical Insights Why Can We Run This on Raspberry Pi 5? Thanks to open-source advancements, we can now run large-scale AI models on small devices like the Raspberry Pi 5. Key factors enabling this include: Optimized lightweight models: DeepSeek-R1 1.5B is built efficiently to run on limited hardware. ARM64 Support: Modern AI frameworks support ARM-based architectures, enabling their use on RPi5. Open-source software: Platforms like Ollama make AI deployment accessible to all. Performance Considerations Running this model on an RPi5 without a GPU will be CPU-intensive. Consider reducing active processes to free up memory. If performance lags, use a lighter model or external processing (cloud inference). No GPU acceleration was used in this setup, meaning all computations rely solely on the CPU, which may affect inference speeds. This guide covers the installation and execution of DeepSeek-R1 1.5B on a Raspberry Pi 5, following the steps demonstrated in your images. ...

Setting Up Neovim: An Easy and Beginner’s Guide Neovim is a modern and extensible text editor that enhances Vim’s capabilities. If you’re using Linux, setting up Neovim can be a rewarding experience, allowing you to customize it for an efficient workflow. In this guide, we’ll cover installing Neovim, setting up a basic configuration, and enhancing it with essential plugins to turn it into a full-fledged IDE. 1. Installing Neovim sudo pacman -S neovim 2. Setting Up Neovim Configuration Neovim’s configuration is stored in ~/.config/nvim/. Create the directory and initialize a basic configuration: ...

Linux Kernal The majority of the kernel’s code is written in C, leveraging extensions provided by the GNU Compiler Collection (GCC) beyond standard C. Additionally, it includes assembly code for architecture-specific functions, such as optimizing memory usage and task execution. Architecturally, the Linux kernel is monolithic, meaning the entire OS operates within kernel space. However, it features a modular design, allowing software components to be integrated as modules, including dynamic loading. ...

What is it? gh is GitHub’s official command-line tool designed to extend Git’s functionality with GitHub-specific features. Purpose: Simplifies interaction with GitHub’s ecosystem directly from the terminal. Allows you to manage repositories and use GitHub features like issues, pull requests, and workflows. Key Features: GitHub-specific tasks: Authentication: Easier login (gh auth login) without dealing with tokens manually. Repository Management: Create, fork, or clone repositories. Issues & Pull Requests: Manage issues, PRs, and comments directly. Actions: Manage and view GitHub Actions workflows. Works alongside Git for basic version control tasks. Use Case: Best for developers heavily using GitHub and its features (for example: pull requests, issues, and actions). ...

Connecting a Raspberry Pi 5 to a USB TTY cable is a common way to interact with it through a serial connection, especially for debugging or setting up the device without using a display. Prerequists Raspberry Pi 5. USB TTY (serial) cable. Computer with a terminal emulator (minicom/screen). GPIO pinout diagram of Raspberry Pi 5 (for reference). Power source for Raspberry Pi (optional if USB TTY can power it, though not recommended). Before Starting! Issuse with firmware UART does NOT work on the RPI5 from the factory. We will need a firmware update to fix this that prevents the dtoverlays for UARTs from working. ...

Hosting a website on GitHub Pages with Hugo involves the following steps: Creating a website 1. Install Hugo and git > sudo pacman -S Hugo 2. Create a new Hugo site > hugo new site your-website 3. Add a Theme Navigate to your website directory and add a theme. You can choose one from the Hugo Themes . > cd your-website > git init > git submodule add https://github.com/adityatelange/hugo-PaperMod.git themes/hugo-PaperMod Now you will need to update the hugo.toml file for them to take effect. To do so you can either echo or addd it in the file. ...

KVM Kernel-based Virtual Machine is a free and open-source virtualization module in the Linux kernel that allows the kernel to function as a hypervisor. Installation For updates, run the following command: $ sudo pacman -Syu QEMU/KVM installation: We’ll install qemu and all the utils required: $ sudo pacman -S qemu vde2 ebtables iptables-nft nftables dms masq bridge-utils ovmf swptm Virtual Machine Manager installation: The virt-manager application is a graphical user interface for managing virtual machines through libvirt. It primarily targets KVM VMs. ...