[{"content":"\nSome time ago I vibe-coded a project management app (which i still use today) because pretty much all the existing options like Monday, Asana, and MS Project suck in one way or another, or paywall the features I needed. I remember searching for the easiest, most painless way to deploy it and it turned out containers were the answer. Super straightforward.\nDocker or Podman? There are many options to run containers but it usually comes down to Docker or Podman.\nDocker works great, but Docker needs a daemon (dockerd) running at all times, and I don\u0026rsquo;t want another background service in my system. For a personal server I don\u0026rsquo;t fully trust myself, so the less the better.\nPodman doesn\u0026rsquo;t have a daemon. You run podman run and the container is just a process. That\u0026rsquo;s a model I like.\nIf you\u0026rsquo;re on Fedora or any RHEL-based distro, Podman is already installed. If not, just install it from source or your distro repo. If you already use Docker, Podman will work the same, same syntax, registries, and images work.\nYou can even alias docker to podman:\nalias docker=podmanQuadlets vs Compose For running a full stack app with multiple services you have: podman-compose (or docker-compose) or Quadlets.\npodman-compose Most guides and tutorials show you how to run stacks with docker-compose and podman-compose works the same.\nThis works fine. The downside is that there are some \u0026ldquo;features\u0026rdquo; that may not work as well in podman-compose and something could silently break. Also every time you want automated restarts, update hooks, or boot persistence, you\u0026rsquo;re bolting things on top. All the \u0026ldquo;how do I make this run on reboot\u0026rdquo; guides in the Docker world are basically gluing compose to systemd with duct tape. It sucks.\nQuadlets Quadlets are a way to run Podman containers with native systemd integration. You only have to write a .container file and reload the systemd daemon to run the container as a systemd service. Natively.\nThe only thing that sucks is that every \u0026ldquo;component\u0026rdquo; (container, network, volume) needs its own file, so multi-service stacks get verbose and clunky. There are also way fewer guides for this compared to compose, which is annoying. But it\u0026rsquo;s a one-time setup and you never have to think about it again.\nTo use Quadlets, create a .container file in ~/.config/containers/systemd/ (for rootless containers):\n[Container] Image=localhost/myapp:latest PublishPort=3000:3000 Environment=NODE_ENV=production [Service] Restart=always [Install] WantedBy=default.targetThen reload the systemd daemon to implement the changes:\nsystemctl --user daemon-reload systemctl --user enable --now myappNow your app starts on boot, restarts on failure, and logs go to the journal (you can even configure it to pull updates automatically from a container registry):\njournalctl --user -u myapp -fAgain, the thing that I love about Quadlet is that you get the \u0026ldquo;pain\u0026rdquo; (not really) in the initial setup only and forget about it.\nExposing Your App With Caddy If you\u0026rsquo;re using Caddy as a reverse proxy (which I recommend), nothing special is needed. Container exposes a port, Caddy proxies to it, business as usual:\nmyapp.yourdomain.com { reverse_proxy localhost:3000 }One thing: rootless containers can\u0026rsquo;t bind to ports below 1024. Expose something in the 3000s and let Caddy handle 80 and 443.\nThe SELinux Pain If you are using SELinux, you might get some errors when running containers. Keep it enabled though.\nVolume Mounts: SELinux silently blocks containers from reading mounted host directories. To fix it just add :Z to your volume mounts:\npodman run -v ./data:/app/data:Z myappRootless Permission Errors: Some images were built assuming they run as root and throw weird permission errors in rootless mode. Fix it with --userns=keep-id, which maps your UID into the container so file permissions line up:\npodman run --userns=keep-id -v ./data:/app/data:Z myappTo Recap? Create a vibe-coded app, run it as container, build the image, drop a Quadlet file, enable the \u0026ldquo;container\u0026rdquo; service, point Caddy at the port, access the app from the exposed port or DNS name (if configured). No Docker daemon, no root processes, no compose files.\nI\u0026rsquo;ve been running all my personal projects this way and once the workflow is setup it becomes very easy to maintain, secure, and rollouts become super easy and fast. Maybe I\u0026rsquo;ll do more posts about Podman containers? Maybe not?\n","permalink":"https://danoss.me/en/categories/linux/podman-or-docker-containers/","summary":"Choosing Podman or Docker containers for deploying vibe-coded apps","title":"Podman or Docker Containers?"},{"content":"\nFor some fucking reason I now have to manage MDM for many companies. So figured I\u0026rsquo;ll have to learn more in depth how they work\u0026hellip;\nSome companies give employees a \u0026ldquo;work phone\u0026rdquo; or ask to download software to use work applications in their personal phones. The thing is that is not just giving access to company resource, but also allows admins/IT to view much more details of your phone and its usage. Kinda like a spyware.\nMDM (Mobile Device Management) is like an invisible layer between the hardware (phone) and the company/org IT team. There are many reasons why a company would want to implement MDM, such as security compliance, remote data wipe when devices get lost, enforce encryption, monitor correct, etc. However it allow allows employers to view the employee behavior up to a surprising level of detail.\nMany people has iOS devices but I\u0026rsquo;ll focus on Android, because fuck Apple.\nWhat is MDM Software? MDM (Mobile Device Management) is not spyware. It doesn\u0026rsquo;t record your microphone or screenshot your screen on demand. It\u0026rsquo;s a framework: a combination of device-level agents, cloud policy servers, and OS-level APIs that let IT administrators configure, monitor, and control enrolled devices at scale.\nThe major platforms are Microsoft Intune, VMware Workspace ONE, and IBM MaaS360. All of them connect to the same underlying OS APIs, they pretty much differ in features, not in fundamental capability.\nThe most important thing to understand is that what MDM can see depends on how the device was enrolled. A company-issued phone enrolled in full device management mode is more of a privacy nightmare than a personal phone enrolled in a BYOD work profile.\nWhat Your Employer Can Actually See On a company-owned, fully managed device, the employer\u0026rsquo;s MDM platform has device-owner-level access to the Android OS. That includes:\nEvery app installed on the device Real-time and historical GPS location Battery level Wi-Fi networks the device has joined Mobile data and Wi-Fi traffic volume per app Device compliance status (rooted? screen lock set? OS patched?) Browsing history, when a managed browser profile is deployed On a personal phone enrolled via Android Work Profile (the standard BYOD setup), the profile is isolated container running under a separate user ID. The MDM is scoped to profile owner permissions and cannot reach into the personal side of the device. Personal apps, photos, messages, and browsing history are invisible to IT.\nHow Android MDM Works The Three Management Modes Google\u0026rsquo;s Android Enterprise framework has 3 management modes, each granting different levels of OS access to the deployed DPC (Device Policy Controller) app.\nflowchart TB subgraph DO[\u0026#34;FULLY MANAGED: Device Owner\u0026#34;] D1[\u0026#34;DPC = Device Owner\u0026lt;br\u0026gt;Full OS control, work-only device\u0026#34;] D2[\u0026#34;Sees: all installed apps\u0026lt;br\u0026gt;GPS location\u0026lt;br\u0026gt;Per-app network usage\u0026lt;br\u0026gt;Device-wide policy enforcement\u0026#34;] D1 --\u0026gt; D2 end subgraph COPE[\u0026#34;WORK PROFILE ON COMPANY-OWNED DEVICE, Android 11+\u0026#34;] C1[\u0026#34;DPC = Profile Owner\u0026lt;br\u0026gt;with elevated company-owned policies\u0026#34;] C2[\u0026#34;Work Profile\u0026lt;br\u0026gt;IT manages fully: apps, data, policies\u0026#34;] C3[\u0026#34;Personal Side\u0026lt;br\u0026gt;App content invisible to IT\u0026lt;br\u0026gt;Device-wide security policies enforced\u0026#34;] C1 --\u0026gt; C2 C1 --\u0026gt; C3 end subgraph BYOD[\u0026#34;WORK PROFILE: Personally Owned Device\u0026#34;] B1[\u0026#34;DPC = Profile Owner only\u0026#34;] B2[\u0026#34;Work Profile\u0026lt;br\u0026gt;IT manages fully\u0026#34;] B3[\u0026#34;Personal Side\u0026lt;br\u0026gt;OS-enforced kernel barrier\u0026lt;br\u0026gt;IT has zero access\u0026#34;] B1 --\u0026gt; B2 B2 -. \u0026#34;blocked, separate Linux UID\u0026#34; .-\u0026gt; B3 end style DO fill:#1a2e1a,stroke:#4caf50,color:#e8f5e9 style COPE fill:#1a1a2e,stroke:#5c7cfa,color:#e8eaf6 style BYOD fill:#2e1a1a,stroke:#ef5350,color:#fce4ec Fully Managed (Device Owner): The company owns and controls the entire device.\nWork Profile on Company-Owned Device The DPC acts as a Profile Owner with access to extended company-owned device policies via getParentProfileInstance(). IT fully manages the work profile and can enforce device-wide security policies like screen lock and encryption, but personal app data remains inaccessible.\nWork Profile on Personally-Owned Device (BYOD): This is the strictest privacy separation. IT cannot see the personal profile. Yay!\nHow Device Owner Status Is Granted Device Owner mode can only be set on a factory-reset or a brand new phone. The provisioning process is orchestrated by the Android setup wizard, which grants Device Owner status to the specified DPC package through an internal system call. The enrollment methods are:\nflowchart LR subgraph ZT[\u0026#34;Zero-Touch\u0026#34;] ZT1[\u0026#34;Device purchased\u0026lt;br\u0026gt;from authorized reseller\u0026#34;] --\u0026gt; ZT2[\u0026#34;IT pre-registers\u0026lt;br\u0026gt;IMEI in ZT portal\u0026#34;] ZT2 --\u0026gt; ZT3[\u0026#34;First boot contacts\u0026lt;br\u0026gt;Google ZT servers\u0026#34;] ZT3 --\u0026gt; ZT4[\u0026#34;DPC APK downloaded\u0026lt;br\u0026gt;and installed silently\u0026#34;] end subgraph QR[\u0026#34;QR Code\u0026#34;] QR1[\u0026#34;Factory-reset device\u0026#34;] --\u0026gt; QR2[\u0026#34;Tap setup screen\u0026lt;br\u0026gt;6 times in same spot\u0026#34;] QR2 --\u0026gt; QR3[\u0026#34;Scan QR code from\u0026lt;br\u0026gt;IT console\u0026#34;] QR3 --\u0026gt; QR4[\u0026#34;JSON payload decodes:\u0026lt;br\u0026gt;DPC package, APK URL,\u0026lt;br\u0026gt;enroll token, WiFi creds\u0026#34;] QR4 --\u0026gt; QR5[\u0026#34;Android downloads\u0026lt;br\u0026gt;and installs DPC APK\u0026#34;] end subgraph AFW[\u0026#34;DPC Identifier\u0026#34;] AFW1[\u0026#34;Factory-reset device\u0026#34;] --\u0026gt; AFW2[\u0026#34;Enter afw#setup\u0026lt;br\u0026gt;instead of Google account\u0026lt;br\u0026gt;at setup wizard\u0026#34;] AFW2 --\u0026gt; AFW3[\u0026#34;Wizard downloads DPC\u0026lt;br\u0026gt;from Play Store\u0026#34;] end subgraph KME[\u0026#34;Knox Mobile Enrollment\u0026lt;br\u0026gt;Samsung only\u0026#34;] KME1[\u0026#34;Register device IMEI\u0026lt;br\u0026gt;or serial in Knox portal\u0026lt;br\u0026gt;(no reseller required)\u0026#34;] --\u0026gt; KME2[\u0026#34;Device pulls config\u0026lt;br\u0026gt;and DPC on first boot\u0026#34;] end subgraph FINAL[\u0026#34;Provisioning completes\u0026#34;] F1[\u0026#34;DPC installed\u0026#34;] --\u0026gt; F2[\u0026#34;Setup wizard grants\u0026lt;br\u0026gt;Device Owner status\u0026lt;br\u0026gt;to DPC package\u0026#34;] F2 --\u0026gt; F3[\u0026#34;Device enrolls with\u0026lt;br\u0026gt;MDM cloud server\u0026#34;] F3 --\u0026gt; F4[\u0026#34;Policy pushed:\u0026lt;br\u0026gt;CA certs, VPN config,\u0026lt;br\u0026gt;app restrictions,\u0026lt;br\u0026gt;location reporting...\u0026#34;] end ZT --\u0026gt; FINAL QR --\u0026gt; FINAL AFW --\u0026gt; FINAL KME --\u0026gt; FINAL style ZT fill:#1a2e1a,stroke:#4caf50,color:#e8f5e9 style QR fill:#1a1a2e,stroke:#5c7cfa,color:#e8eaf6 style AFW fill:#2e1f1a,stroke:#ffa726,color:#fff3e0 style KME fill:#2e2e1a,stroke:#cddc39,color:#f9fbe7 style FINAL fill:#1a1a1a,stroke:#ef5350,color:#fce4ec The SSL Inspection Problem When an MDM pushes a corporate root CA certificate, the OS adds it to the system trust store. Every TLS connection that validates against the system trust store (which is most of them) now trusts certificates signed by that CA. Combined with setAlwaysOnVpnPackage(lockdownEnabled = true), the company routes all device traffic through a corporate proxy that performs full TLS break-and-inspect:\nflowchart TB subgraph PREREQ[\u0026#34;MDM Pre-conditions Active\u0026#34;] S1[\u0026#34;installCaCert\u0026lt;br\u0026gt;Corp root CA pushed to system trust store\u0026#34;] S2[\u0026#34;setAlwaysOnVpnPackage lockdown=true\u0026lt;br\u0026gt;All traffic routed through org proxy\u0026#34;] end subgraph INTERCEPT[\u0026#34;Standard HTTPS Request: Intercepted\u0026#34;] N1[\u0026#34;App sends TLS ClientHello to example.com\u0026#34;] N2[\u0026#34;Corporate proxy intercepts the connection\u0026#34;] N3[\u0026#34;Proxy opens separate TLS session to example.com\u0026lt;br\u0026gt;Receives real DigiCert certificate\u0026#34;] N4[\u0026#34;Proxy presents forged cert to app\u0026lt;br\u0026gt;Corp CA signed example.com\u0026lt;br\u0026gt;App trusts Corp CA, no warning shown\u0026#34;] N5[\u0026#34;App request decrypted by proxy\u0026lt;br\u0026gt;Logged in IT console\u0026lt;br\u0026gt;Re-encrypted and forwarded to server\u0026#34;] N6[\u0026#34;Server response decrypted by proxy\u0026lt;br\u0026gt;Logged in IT console\u0026lt;br\u0026gt;Re-encrypted back to app\u0026#34;] N1 --\u0026gt; N2 --\u0026gt; N3 --\u0026gt; N4 --\u0026gt; N5 --\u0026gt; N6 end subgraph PINNED[\u0026#34;Certificate-Pinned Apps: Not Interceptable\u0026#34;] P1[\u0026#34;App sends TLS ClientHello to Signal servers\u0026#34;] P2[\u0026#34;Proxy presents Corp CA signed certificate\u0026#34;] P3[\u0026#34;App checks cert fingerprint against hardcoded value\u0026lt;br\u0026gt;Fingerprint mismatch, connection refused\u0026lt;br\u0026gt;No silent interception possible\u0026#34;] P1 --\u0026gt; P2 --\u0026gt; P3 end PREREQ --\u0026gt; INTERCEPT PREREQ --\u0026gt; PINNED style PREREQ fill:#1a1a2e,stroke:#5c7cfa,color:#e8eaf6 style INTERCEPT fill:#2e1a1a,stroke:#ef5350,color:#fce4ec style PINNED fill:#1a2e1a,stroke:#4caf50,color:#e8f5e9 The proxy decrypts traffic, inspects it for DLP policy violations, logs it, re-encrypts it with a Corp CA-signed certificate, and forwards it. From the app\u0026rsquo;s perspective, TLS succeeded. From IT\u0026rsquo;s perspective, they saw plaintext.\nWhat Can I Do? Certificate pinning. Apps that hardcode expected certificate fingerprints (like Signal) validate the certificate chain independently of the OS trust store. If the fingerprint doesn\u0026rsquo;t match what the app expects, the connection fails rather than silently accepting the proxy\u0026rsquo;s certificate. Signal\u0026rsquo;s traffic appears to the corporate proxy as an opaque TLS stream to Signal\u0026rsquo;s servers; the proxy can\u0026rsquo;t decrypt it and can\u0026rsquo;t provide a plausible forged certificate that Signal would accept.\nCheck Your Android Device\u0026rsquo;s Enrollment State Steps vary depending on brand/OEM, but these are for Samsung.\nCheck if MDM is Active:\nAny app listed here holds elevated device administration permissions Settings → Security and privacy → More security settings → Device admin appsCheck the Management Mode:\nA Work Profile appears as a separate section with a briefcase badge on work-side app icons. If there\u0026rsquo;s no work profile listed but device admin apps are present, you\u0026rsquo;re on a fully managed device with no personal/work separation Settings → Accounts and backup → Manage accountsCheck for Pushed CA Certificates:\nAny certificate in the User tab that you didn\u0026rsquo;t install yourself was pushed by an MDM On a fully managed device also check the System tab, since Device Owner DPCs can install CAs at system level where they\u0026rsquo;re less visible Settings → Security and privacy → More security settings → View security cetificated → User tabCheck for Always-on VPN:\nA VPN profile with a lock icon and no option to disconnect is an MDM-enforced always-on VPN. Every packet you send goes through corporate infrastructure regardless of which network you\u0026rsquo;re on, including cellular Settings → Connections → More connection settings → VPNWhat You Can Do I don\u0026rsquo;t like MDM-controlled devices. The most effective protection is to have a separate device for work and another for personal use (never mix them, technology improves and becomes more anti-privacy). Using a company phone for personal activity is effectively consenting to the monitoring, whether or not you\u0026rsquo;re aware of the full extent.\nIf you want to communicate with another person, use a private personal device and use Signal. Certificate pinning prevents the corporate SSL inspection proxy from intercepting Signal traffic. End-to-end encryption means the content is protected regardless. The MDM can see Signal is installed and that traffic is flowing to Signal\u0026rsquo;s servers, it cannot see what you\u0026rsquo;re saying.\nDon\u0026rsquo;t log into personal accounts on a managed device using the corporate browser or any app without certificate pinning. On an always-on VPN deployment, assume the proxy sees plaintext for those sessions.\nFinally, read your company\u0026rsquo;s BYOD agreement. You will always be fucked, but no one cares i guess\u0026hellip;\n","permalink":"https://danoss.me/en/categories/others/mdm-and-privacy/","summary":"MDM on your phones and what exactly companies know about you? Good? Bad? Middle?","title":"MDM and Privacy???"},{"content":"Now something related to the previous Signal post.\nUsually someone needs physical access to your phone to see your data, however that is not usually needed.\nFCM Tokens for Surveillance Nowadays many governments request push notification records from Apple and google for their investigations. You don\u0026rsquo;t need to be a criminal to be targeted, just ask UK people.\nThe US government specially has been building a gigantic surveillance machine that gathers many data points and metadata not only from US citizens but from the whole world, and this includes push notification data from all phones.\nAll applications (even Signal) use push notifications, so we are all fucked.\nWhen you install Signal and grant notification permissions on your phone, GMS registers your devices with FCM, which produces a unique FCM registration token (like a delivery address for your device-app account combination). This token is sent to Signal\u0026rsquo;s servers so they know where to your wake-up pings, all of this is normal.\nThe shady part is that Google also stores its own copy of this mapping. This FCM token belongs to this Google account, on this device, identified by its Android ID and hardware identifiers. In other words, from that moment on, every single Signal notification delivery event creates a log entry at Google\u0026rsquo;s infrastructure: the target FCM token, Signal\u0026rsquo;s app identifier (org.thoughtcrime.securesms), and a delivery timestamp.\nUsually for \u0026ldquo;normal\u0026rdquo; applications, the text message content of the notification is also stored in Google\u0026rsquo;s servers, but in the case of Signal, only data-only FCM messages with empty payloads. Google doesn\u0026rsquo;t need the whole message content to track you, they know you got a message, time, etc. It\u0026rsquo;s all about metadata.\nThe deanonymization chain works like this. Law enforcement identifies a person of interest who uses Signal under an anonymous account. They have a device identifier, or an app usage pattern, or a tip pointing to a specific FCM token. With a court order to Google, law enforcement now has the full token mapping. The FCM token resolves to a Google account. The Google account resolves to a real identity through payment records, a recovery phone number, a recovery email, or simply the name the account was registered under. The \u0026ldquo;anonymous\u0026rdquo; Signal user is now a named individual with a timestamped history of when they received Signal messages (not the messages though, but you are not anonymous).\nNone of this requires accessing the target\u0026rsquo;s device. None of it requires breaking Signal\u0026rsquo;s encryption. It is purely a metadata problem, and the metadata lives at Google, and people had been killed by metadata.\nDo Not Enable Notifications? The token exists the moment you grant notification permission\u0026hellip;\nIn Signal\u0026rsquo;s case, the FCM token is generated and registered with Google\u0026rsquo;s infrastructure the first time GMS processes a notification registration from Signal, which happens as soon as Signal is installed and you grant notification permissions. You do not need to receive any messages. You do not need to send any. The token exists at Google from the moment Signal\u0026rsquo;s FCM registration call completes. Every notification delivery after that adds a timestamped record to Google\u0026rsquo;s logs.\nThis is not a Signal-specific problem. Every app on your Android device that uses FCM (which is most of them if not all) generates a token that Google holds and maps to your Google account.\nSilent Push: When Your Phone Gets Pinged to Reveal Your Location Beyond the FCM surveillance trail, governments send a silent push notification, which is a high-priority FCM data message with no notification payload and no visible content (can be sent to any device with a valid FCM token). The device receives it, GMS wakes the target app via onMessageReceived(), and the app does whatever it is coded to do in response. Nothing appears on your screen. No banner, no sound, no badge. Your phone just quietly woke up in the background.\nThe surveillance value comes from what happens when the woken app makes a network connection. If the app connects to any server in response to the wake-up to check for updates, sync state, etc., and that server logs your device\u0026rsquo;s current IP address (which resolves to your location).\nThe silent push is used by:\nMarketing and analytics platforms. For example if an app is uninstalled the FCM token returns a NOT_REGISTERED error, which means the device no longer has the app. For law enforcement use, the path runs through an app developer. An investigator with a target\u0026rsquo;s FCM token and a court order compelling the app developer to cooperate can have the developer send a silent push to that specific token. The app wakes, connects to its servers, the server logs the IP. Alternatively, Google\u0026rsquo;s FCM infrastructure itself records connection events from the target device in response to the delivery, revealing the IP to Google regardless of what the app does. So, every Android app using FCM has the same structure: a token at Google, mapped to your account, generating a delivery log with every notification.\nSo Now What? If using Signal, changing the notification content setting to \u0026ldquo;No Name or Content\u0026rdquo; to close off the device forensics threat. The string \u0026quot;Signal message\u0026quot; ends up in the database instead of sender names and message text. Anyone extracting that database finds nothing useful.\nStill Google will have metadata though. Your FCM registration token exists at Google regardless of what your notification content setting says. Delivery timestamps are logged at Google regardless.\nThe only thing that removes the FCM token from Google\u0026rsquo;s records is never having registered with FCM in the first place.\n","permalink":"https://danoss.me/en/categories/security/push-notifications-can-track-you/","summary":"How push notifications can track you without touching your phone","title":"Push Notifications Can Track You"},{"content":"This story got me thinking: https://www.404media.co/fbi-extracts-suspects-deleted-signal-messages-saved-in-iphone-notification-database-2/\nThe FBI recovered Signal messages from a suspect\u0026rsquo;s phone, Signal\u0026rsquo;s encryption was cracked, nothing is safe, etc. But that\u0026rsquo;s not it. Signal\u0026rsquo;s encryption was not broken. What the FBI actually did was pull decrypted message content from Apple\u0026rsquo;s internal notification storage on the suspect\u0026rsquo;s iPhone, which had nothing to do with Signal\u0026rsquo;s cryptography and everything to do with how phones handles push notifications.\nEven though this case happened on an iPhone, the problem is not iOS-exclusive. By default, both Android and iOS write decrypted notification messages to system databases that persist through app deletion and disappearing message settings. That data is accessible to anyone with physical access to your device.\nWhat Actually Happened In April 2026, 404 Media reported on an FBI agent testimony during a federal trial involving defendants charged with activities at an ICE detention facility in Texas. One defendant had Signal installed on her iPhone at the time of the events. By the time investigators examined the phone, Signal had been deleted.\nThey recovered Signal messages anyway, specifically incoming ones, through Apple\u0026rsquo;s internal notification storage. The app was gone but the messages were not.\nHow Signal Works on iOS The FBI case involved an iPhone, so here\u0026rsquo;s how iOS handles push notification storage and Signal specifically.\nAPNs: Apple\u0026rsquo;s Centralized Push Infrastructure Apple Push Notification Service (APNs) is the only way to deliver background notifications to iPhones. Every remote notification for every app must go through Apple\u0026rsquo;s servers before it reaches the device.\nWhen an app registers for push notifications, the device gets an APNs device token, a unique identifier Apple uses to route notifications to that specific device. Signal\u0026rsquo;s servers hold this token and use it to request notification delivery from Apple. From that point on, Apple is a mandatory intermediary for every notification Signal sends.\nNSE: The Notification Service Extension Signal on iOS uses a UNNotificationServiceExtension (NSE), a sandboxed process that iOS fires between receiving an APNs notification and displaying it to the user. This is like a middleware that intercepts the notification before it ever shows up on your screen.\nHere is what the flow looks:\nSignal\u0026rsquo;s server sends an APNs payload to Apple containing 2 things: a mutable-content: 1 flag, which tells iOS to run the NSE before displaying anything, and an encrypted ciphertext blob containing the actual message content. Apple routes this payload to the target device. The message content (even if encrypted), travels through Apple\u0026rsquo;s infrastructure. Once the payload arrives on device, iOS fires Signal\u0026rsquo;s NSE in a separate sandboxed process to decrypt the ciphertext locally using Signal Protocol keys and calls contentHandler() with the plain-text readable sender name and message text. iOS takes that decrypted output, stores it in the phone\u0026rsquo;s notification database at /var/mobile/Library/UserNotifications/, and displays the notification. That local database is what the FBI extracted. It persists through app deletion, message expiry, and every disappearing message timer you have ever set.\nHow Signal Works on Android Android handles this entirely differently:\nWhat Google Actually Sees via FCM Android does not allow apps to maintain persistent background connections to their own servers. Battery optimizations, App Standby, and background process limits all prevent it. Instead, Google centralizes all push delivery through a single persistent connection managed by Google Mobile Services (GMS).\nGMS keeps an always-on TCP socket to Firebase Cloud Messaging (FCM) servers that survives across all power states short of the device being off. Every push notification for every app goes through this channel.\nFCM supports 2 message types:\nNotification messages carry a title and body directly in the FCM payload. Android can render the notification from that payload without ever waking the app. The content travels through Google\u0026rsquo;s FCM infrastructure. Google\u0026rsquo;s servers carry the notification text. Data messages are what Signal uses. They carry only a custom key-value payload and require the receiving app to handle it in code. Signal\u0026rsquo;s data message payload is essentially empty, a high-priority wake-up ping with no message content whatsoever. The actual text of your message is never in the FCM payload at any point in transit. All Google\u0026rsquo;s infrastructure ever sees is a target device FCM registration token, Signal\u0026rsquo;s app identifier (org.thoughtcrime.securesms), a priority flag, and a timestamp. Nothing else.\nThis is the contrast to iOS. On iOS, Apple\u0026rsquo;s APNs infrastructure carries an encrypted blob. On Android, Google\u0026rsquo;s FCM carries nothing related to your message content at any point in transit, not even ciphertext.\nWhat Google does have is delivery metadata: your device\u0026rsquo;s FCM registration token, which maps to your Google account, and a timestamped record of when a Signal notification was delivered to your device (which is useful to track you so\u0026hellip;).\nThe Double Ratchet: Where Decryption Happens When GMS receives the FCM data message and wakes Signal\u0026rsquo;s FirebaseMessagingService, Signal gets roughly a 20-second execution window. It uses that window to open a TLS connection to Signal\u0026rsquo;s own servers and fetch the actual encrypted message payload.\nThe encryption is the Signal Protocol, which combines 2 mechanisms:\nX3DH (Extended Triple Diffie-Hellman) handles the initial key agreement when 2 parties establish a conversation for the first time. The Double Ratchet algorithm derives a new encryption key for every individual message using both a symmetric-key ratchet and a Diffie-Hellman ratchet. This means past messages remain encrypted even if a current key is compromised, because each ratchet step generates a new key and discards the old one. Future messages also become secure again once the ratchet advances after a compromise. Signal\u0026rsquo;s servers store only encrypted ciphertext and cannot decrypt any of it. Signal cannot read your messages. Decryption happens entirely on your device, with private keys living in Android\u0026rsquo;s hardware-backed Keystore on devices with a secure element.\nThe Handoff: Where the OS Takes Over Once Signal decrypts the message and needs to show it to you, it calls Android\u0026rsquo;s NotificationManager. Stripped down, that call looks like this:\nval notification = NotificationCompat.Builder(context, CHANNEL_ID) .setContentTitle(senderName) .setContentText(messageBody) .setSmallIcon(R.drawable.ic_notification) .build() NotificationManagerCompat.from(context).notify(notificationId, notification)The moment Signal calls .notify(), the senderName and messageBody leave Signal\u0026rsquo;s sandboxed process and are handed to a system service. The Android OS takes ownership of it for rendering: the lock screen, the notification drawer, connected Wear OS devices, and system-level logging.\nIf you configure Signal\u0026rsquo;s notification content setting to \u0026ldquo;No Name or Message\u0026rdquo;, what gets passed to .setContentTitle() and .setContentText() is a generic placeholder. The OS takes ownership of that placeholder instead, and nothing sensitive ends up in any database.\nWhere Android Stores Your Notifications The System Notification Log When Signal calls NotificationManager.notify(), Android writes the notification event to a SQLite database at /data/system/notification_log.db. This database is owned by the OS, not by Signal. Signal has no visibility into it and no way to delete entries from it. From Signal\u0026rsquo;s perspective, it does not know the database exists.\nThe schema stores: pkg (the posting app\u0026rsquo;s package name, e.g. org.thoughtcrime.securesms), uid, when (Unix timestamp in milliseconds), tag, key, and the content fields containing whatever strings Signal passed to .setContentTitle() and .setContentText() when it called notify(). If those strings were a sender name and message body, that is exactly what the database holds.\nThis database is invisible to users. There is no Settings menu that exposes it and no way to manually clear it through the Android interface. Accessing it requires physical device extraction via forensic tools.\nDeleted Notifications Are Not Actually Gone SQLite\u0026rsquo;s deletion behavior is the reason this database functions as a forensic artifact.\nWhen SQLite deletes a row, it does not overwrite the bytes on disk. It marks the pages holding that row as free and available for reuse. The data remains there until SQLite decides to allocate those pages for new writes.\nForensic tools parse the database at the page level, reading \u0026ldquo;deallocated\u0026rdquo; pages to recover deleted row data. This content typically remains recoverable for weeks or more, until enough new writes physically displace it.\nThe FCM Queued Messages Store Separately, undelivered FCM messages are queued in a LevelDB database at /data/data/com.google.android.gms/databases/fcm_queued_messages.ldb/, used by Play Services to buffer FCM payloads that arrived while the app was not running.\nFor most apps using notification-type FCM messages, this store can contain actual notification content. For Signal, it is less interesting forensically. Because Signal uses data-only FCM messages with empty payloads, the queued message store contains nothing but wake-up pings and delivery timestamps. Useful metadata (which to be fair, people have been killed from just metadata), but not message content.\nExpand image What You Should Do What you should do depends on your paranoia and how hard you want your life to be. The below focuses on Android because I\u0026rsquo;m more familiar with it.\n1. Change Signal\u0026rsquo;s Notification Settings Open Signal -\u0026gt; Settings -\u0026gt; Notifications -\u0026gt; Show, and set it to \u0026ldquo;No Name or Message\u0026rdquo; This controls what strings Signal passes to NotificationManager, which determines what ends up in notification_log.db. With this change, message content is never written to the notification database in the first place. 2. Remove Signal\u0026rsquo;s Notification Permissions Entirely Open Android Settings -\u0026gt; Apps -\u0026gt; Signal -\u0026gt; Notifications -\u0026gt; Off. This removes Signal\u0026rsquo;s notification permissions at the OS level, which causes GMS to deregister Signal\u0026rsquo;s FCM token and remove it from Google\u0026rsquo;s records. Signal falls back to WebSocket polling instead. Messages will arrive within seconds to a minute rather than instantly, but this also eliminates the FCM exposure vector entirely. 3. Use a VPN Always route your traffic through a trusted VPN. Any server your device connects to (including Google\u0026rsquo;s FCM servers) will see the VPN endpoint IP rather than your real one. Removing Signal\u0026rsquo;s notification permissions as above also eliminates this exposure vector, but a VPN adds a layer of protection across everything else running on your device. 4. Use GrapheneOS Use GrapheneOS to go a step further. GrapheneOS is a hardened Android fork built for Google Pixel hardware that runs without any Google tokens or infrastructure. No GMS, no FCM, no pinging Google\u0026rsquo;s servers. Signal on GrapheneOS can use its own WebSocket-based push delivery without any Google involvement whatsoever, which removes this concern altogether. Note that everything changes constantly but this post should give you an idea on what to do no only on Signal, but any other app. As for the topic at hand, Signal\u0026rsquo;s encryption is not the weak link here. The weak link is the infrastructure your OS relies on to show you a notification and what it does with that content afterward.\n","permalink":"https://danoss.me/en/categories/security/signal-got-cracked-no/","summary":"Research on the news that Signal messaging app got cracked. Short answer: No","title":"Signal Got Cracked? No"},{"content":"This is a 100% true story, no point in making this shit up. I\u0026rsquo;ve changed some non-important details to avoid identifying the person involved out of respect.\nWhat Happened? I was on a couch talking and drinking with a friend, watching TV. As the night went on and we got tired, we kept talking. At some point I got a little sleepy but still conscious, so I closed my eyes for about 5 seconds and leaned back to rest a bit.\nWhen I opened my eyes I saw my friend\u0026rsquo;s back covered in what looked like a skin-tight gray mesh (like a full-body suit, or one of those swimmer compression suits). Keep in mind he was wearing a red tank top, so this was super weird (like the picture below).\nI stared at his back for about 30 seconds trying to process it. He couldn\u0026rsquo;t have changed clothes so quickly, maybe I was hallucinating? Maybe he changed and I forgot? I was trying to find an explanation.\nWhen I looked closer at his shoulder I could see his tattoos sitting right on top of the gray scaly mesh. It was his skin. Everything was gray, reptile-kind of texture.\nI didn\u0026rsquo;t think to touch it, which I regret (maybe next time, if there is one). I also didn\u0026rsquo;t think to take a photo, though it probably wouldn\u0026rsquo;t have shown anything anyway.\nMy friend had been talking on a surface-level about a really heavy subject, a traumatic experience that left him sad and anxious, but didn\u0026rsquo;t go deep. I kept my mouth shut about what I was seeing and just listened out of respect.\nWhen the topic moved to something else, the gray scaly skin dissolved and the normal skin tone came back.\nA few minutes later I looked at his face and saw the same reptile gray skin again, but only on half of his face was reptile-like. His facial features were still human though.\nSince we weren\u0026rsquo;t talking about the heavy subject anymore, I was curious if he was still thinking about it. I asked him multiple times if he was holding something back, he didn\u0026rsquo;t want to go there, he was dancing around other traumatic experiences but I knew that was not the source of the dark thought so kept pushing. He actually got scared that I was reading his mind.\nEvery time his thoughts went back to that heavy topic, the reptilian skin patches came back. Every time it passed or he got distracted, they faded (pretty much I was able to know when he went to a dark place mentally).\nI was never scared, just curious of what I was seeing.\nMy Theory on Reptilians Maybe the famous reptilians aren\u0026rsquo;t shapeshifting beings that transform into humans to control the world. From what I saw, they could be entities that manifest through human suffering and use vulnerable people to generate more of it.\nMaybe not all of them are bad. Maybe some are neutral or even good. Maybe they don\u0026rsquo;t exist at all and I was just very tired, but I don\u0026rsquo;t think so (due to the surprise of my friend I was reading his thoughts and do a deep dive on trauma).\nAn interesting thing worth noting is that the reptilian skin wasn\u0026rsquo;t \u0026ldquo;green\u0026rdquo; as depicted online. It was more gray, closer to fish skin than reptile, or somewhere in between. Am I going crazy? Maybe\u0026hellip;\n","permalink":"https://danoss.me/en/categories/others/i-saw-a-reptilian/","summary":"True story of my experience with a reptilian\u0026hellip; or maybe I\u0026rsquo;m going crazy?","title":"I Saw a Reptilian?"},{"content":"\nRevised and improved version of the original post: Xbox Controllers in Fedora Linux\nNormally, Xbox Controllers (in my case, Xbox Core Wireless Gaming Controller - might upgrade in the future) works out of the box. However, if it doesn\u0026rsquo;t for any reason read below.\nIn my case, the Xbox Controller was working normally and stopped working after a major system upgrade. The OS could detect the device (udevadm monitor showed the connection) but Steam didn\u0026rsquo;t recognize any input from the Controller.\nDetecting the Problem The controller pairs successfully via Bluetooth or connects via USB. The kernel sees the device connecting and disconnecting (via udevadm monitor). Steam shows no controller input at all. How Linux Handles Xbox Controllers As everything in Linux, everything is handled through various layers.\nThe Linux Input Stack When you connect a controller to a Linux system, the input goes through various subsystems before it reaches the game. Here\u0026rsquo;s what happens:\nThe controller device connects via a transport layer (USB or Bluetooth) The kernel loads the appropriate driver module (kernel module) for that specific device type. The driver translates the raw device data into standard Linux input events, which are exposed through device nodes in /dev/input. Apps like Steam read those device nodes and pass the data to games. If one of these layers fails (missing driver, wrong permissions, missing udev rules, etc.) the controller will not work, even if it\u0026rsquo;s detected by the kernel.\nUSB vs Bluetooth Connections The driver your Xbox controller uses depends on how you connect it, this is important when knowing where to look when troubleshooting.\nUSB connections use the xpad kernel module (part of the kernel-modules-extra package on Fedora). This is the in-tree Linux kernel driver specifically written for Xbox controllers connected over USB and handles Xbox, Xbox 360, Xbox One, and Xbox Series controllers when plugged in via USB. Bluetooth connections do not use xpad, instead they use the standard Bluetooth HID profile. The kernel handles this through the hid-generic driver ( or the third-party hid-xpadneo driver if installed). Xbox Wireless Adapter (Wi-Fi Direct) is a third option. The proprietary Xbox Wireless Adapter uses Wi-Fi Direct, not Bluetooth, and requires the xone driver (available here) or the older xow daemon. The Input Event Pipeline Once the correct driver loads and processes raw device data, the kernel exposes the controller in /dev/input/:\nThe legacy Joystick API creates /dev/input/jsX devices. This is the old interface. Some older games and tools still use it, but it provides less info about available buttons and axes. The evdev API creates /dev/input/eventX devices. This is the modern interface. It provides more info (button states, axis values, force feedback) and is what most modern games expect. You can see both device nodes for a connected controller:\nls -la /dev/input/by-id/ | grep -i xbox # Legacy Joystick API: # usb-Microsoft_Controller_XXXX-joystick -\u0026gt; ../js0 # evdev API: # usb-Microsoft_Controller_XXXX-event-joystick -\u0026gt; ../event16Both interfaces are created from the same driver. If neither exists, the driver didn\u0026rsquo;t load. If only one exists, something is partially broken. If both exist but your game sees nothing, the problem could be udev rules or the application\u0026rsquo;s input configuration.\nWhy Fedora Upgrades Break Xbox Controller Support Kernel Module Packaging on Fedora Fedora splits kernel modules across multiple packages.\nThe base kernel and kernel-modules packages contain modules needed for most common hardware (storage controllers, network adapters, filesystems, display drivers), while the kernel-modules-extra package contains optional modules for less common hardware, like xpad.\nEach kernel version has its own corresponding kernel-modules-extra package. When you install kernel-modules-extra-6.12.5-200.fc41.x86_64, those modules only work with kernel 6.12.5-200.fc41. Install a new kernel and those modules are useless, you need the matching kernel-modules-extra for the new kernel version.\nWhat Happens During A System Upgrade? During a major Fedora version upgrade (e.g., Fedora 43 to 44), dnf system-upgrade downloads and installs packages for the new release.\nThe new release ships a new kernel. If kernel-modules-extra was explicitly installed (you ran dnf install kernel-modules-extra at some point), DNF should also install the corresponding package for the new kernel, but not always\nIf xpad support was pulled in through a dependency chain that changed between releases, or if the package was installed as a weak dependency, or if there was a conflict during the upgrade transaction, kernel-modules-extra can silently drop out. The system boots fine because xpad isn\u0026rsquo;t essential, its just a gamepad driver, so you\u0026rsquo;ll only notice it when trying to use the controller.\nWhy \u0026ldquo;udevadm monitor\u0026rdquo; Is Misleading The udevadm monitor command shows the controller device connecting, but it doesn\u0026rsquo;t mean it is working. The output only shows USB bus activity (meaning the kernel just detected the device being connected).\nKERNEL[12345.678] add /devices/pci0000:00/.../usb1/1-2 (usb) UDEV [12345.690] add /devices/pci0000:00/.../usb1/1-2 (usb)What you actually want to check is whether a driver bound to the device:\n# Check if xpad claimed the device (USB) lsusb -t | grep xpad # Or check dmesg for driver binding dmesg | grep -i xpadIf xpad isn\u0026rsquo;t loaded, these commands return nothing. The device is visible on the bus but orphaned, no driver to translate its data into input events.\nThe Fix For USB Connections (xpad) Check if the Module is Loaded:\nIf this returns nothing, the module isn\u0026rsquo;t loaded. Next, you need to load the module (if installed) or install it. lsmod | grep xpadInstall the Missing Kernel Module:\nThe xpad module is included in the kernel-modules-extra package. Install this package if not already. sudo dnf install kernel-modules-extraLoad the Module and Verify:\nAfter install, reconnect your Xbox controller (unplug and replug USB) and check: lsmod | grep xpad # Expected output: # xpad 32768 0 # ff_memless 20480 1 xpadIf the Module Doesn\u0026rsquo;t Auto-load, Load it Manually:\n# Manually load the module sudo modprobe xpadMake it Persistent Across Reboots:\nThis tells systemd-modules-load.service to load xpad at boot, regardless of whether a matching device is connected. Normally udev handles this automatically (it loads the correct driver when a matching device is detected), but making it explicit ensures it\u0026rsquo;s always available. # Make sure the module load automatically on boot sudo bash -c \u0026#39;echo \u0026#34;xpad\u0026#34; \u0026gt; /etc/modules-load.d/xpad.conf\u0026#39;For Bluetooth Connections If you\u0026rsquo;re connecting via Bluetooth and the controller pairs but doesn\u0026rsquo;t produce input, the issue could be from hid-generic.\nCheck if the HID Device was Created:\n# Look for Xbox controller HID devices ls /dev/input/by-id/ | grep -i xbox # Or check dmesg for HID binding dmesg | grep -i \u0026#34;xbox\\|045e\u0026#34;For Xbox Series Controllers (firmware 5.x+), Verify uhid is available:\nzgrep UHID /proc/config.gz # Should show: CONFIG_UHID=y or CONFIG_UHID=mIf uhid is compiled as a module, ensure it\u0026rsquo;s loaded:\nlsmod | grep uhidInstall xpadneo for full Bluetooth feature support:\nThe basic hid-generic driver gives you button and stick input over Bluetooth, but no rumble, no battery reporting, and sometimes quirky behavior. For a better experience, install the xpadneo driver:\nsudo dnf install dkms kernel-devel git clone https://github.com/atar-axis/xpadneo cd xpadneo sudo ./install.shThis installs a DKMS module that automatically rebuilds against new kernels. It handles force feedback, battery status, trigger rumble, and various controller quirks that hid-generic ignores.\nUse Cases and Drivers Casual gaming with Steam on Fedora (USB cable): Install kernel-modules-extra, plug in your Xbox controller, open Steam. That\u0026rsquo;s it. The xpad driver handles everything, Steam Input handles the rest. This is the simplest path and the most reliable.\nWireless gaming via Bluetooth: Pair the controller through GNOME Settings or bluetoothctl. Basic input works through hid-generic. For rumble and battery reporting, install xpadneo via DKMS. If you use Secure Boot, you\u0026rsquo;ll need to sign the DKMS module or enroll a Machine Owner Key (MOK).\nCouch gaming with multiple controllers: Each connected controller gets its own /dev/input/eventX and /dev/input/jsX pair. Steam handles multiple controllers natively. The only concern is Bluetooth bandwidth if you\u0026rsquo;re connecting 4 controllers wirelessly — a dedicated USB Bluetooth adapter with BLE support helps.\nRetro emulation (RetroArch, Dolphin, PCSX2): These emulators typically use SDL2\u0026rsquo;s gamepad API, which reads from evdev. If the kernel driver is working and the device is in /dev/input/, emulators should pick it up automatically. RetroArch has its own input driver selection — ensure it\u0026rsquo;s set to use udev (not linuxraw) for evdev access.\nIn the End This post shows more in-depth information on how the Xbox controller interacts with the kernel and steps to troubleshoot issues. A little better than the previous version i think. Hope it helps someone.\n","permalink":"https://danoss.me/en/categories/linux/xbox-controllers-on-fedora-linux-2/","summary":"Troubleshooting Xbox controllers in Fedora Linux (and similar distros) - 2 More info","title":"Xbox Controllers on Fedora Linux 2"},{"content":"Part 3 of 4\u0026hellip;or 5?\nThis part is about how corporate or censorship networks use deep packet inspection (DPI) to detect and block VPN traffic and how to hide the fact you are using a VPN.\nNext part will be more practical with use cases, devices, and specific configuration to circumvent censorship.\nConfiguration and examples here are from Mullvad VPN (WireGuard) with Fedora Linux. In the future, I will add examples using Debian-related distros.\nDNS Leaks You installed Mullvad and the VPN is running, everything seems good. Or not?\nExpand image Even if you have your VPN up and running, a DNS leak can silently undo the privacy you think you have.\nA DNS leak happens when your device\u0026rsquo;s DNS queries bypass the VPN tunnel and are sent to your ISP or other 3rd-party DNS servers, potentially exposing the websites you visit. Your traffic is encrypted, your IP is hidden, but your DNS queries are still mapping your entire browsing session to your ISP\u0026rsquo;s logs.\nWhen you visit a website, your device first resolves the domain name into an IP address. That resolution is a DNS query, and it needs to go somewhere. A properly configured VPN tunnels those queries through the encrypted interface and resolves them on the VPN provider\u0026rsquo;s DNS servers. Your ISP should never see which domains you\u0026rsquo;re hitting.\nWhy DNS Leaks Happen Culprit 1: systemd-resolved\nOn Linux, the culprit is usually systemd-resolved not being configured correctly for the VPN interface.\nsystemd-resolved is a system service built into most modern Linux distros that acts as a centralized DNS resolver for the whole OS. Rather than each application making its own DNS queries directly, they all go through systemd-resolved.\nWhen the Mullvad app connects, it registers its DNS server (10.64.0.1) and the ~. routing domain with systemd-resolved (essentially telling it: \u0026ldquo;for all domain lookups, use this interface\u0026rdquo;). If that registration doesn\u0026rsquo;t happen correctly, systemd-resolved falls back to whatever DNS it got from DHCP, which is typically your router, which is typically your ISP.\nCheck it:\nresolvectl status # Under the wg0-mullvad interface, you want to see: # Current DNS Server: 10.64.0.1 # DNS Domain: ~.If you see ~. under the VPN interface, all DNS queries are routed there. If you don\u0026rsquo;t, you have a leak.\nEven with no leak, you still control which DNS resolver handles your queries inside the tunnel. Mullvad defaults to its own resolver (10.64.0.1), but you may have it set to a custom one like Cloudflare. Both are routed through the VPN, so your ISP sees nothing either way, the only difference is which 3rd-party resolves your domains. Cloudflare is faster and globally distributed, but it\u0026rsquo;s a US-based commercial company. Mullvad\u0026rsquo;s resolver is no-logging, Sweden-based, and already inside your trust boundary since your traffic goes through them anyway. You can switch and verify with:\n# Check your current config mullvad dns get # Switch to Mullvad\u0026#39;s resolver mullvad dns set default # Use the Cloudflare DNS resolver mullvad dns set custom 1.1.1.1 1.0.0.1 # Verify what resolver Mullvad sees (empty = custom DNS) curl https://am.i.mullvad.net/dns # Full connection check: IP, DNS, VPN status curl https://am.i.mullvad.net/jsonCulprit 2: Browser Configuration\nThe second common culprit is your browser. Both Chrome and Firefox support DoH (DNS-over-HTTPS), which lets the browser handle DNS resolution entirely on its own, bypassing the system resolver.\nChrome\u0026rsquo;s Secure DNS runs in automatic mode by default: it tries to upgrade your system-configured DNS provider to DoH if that provider is on its known list (Google, Cloudflare, NextDNS, etc.). If your OS is already pointed at Google DNS, Chrome will resolve via 8.8.8.8 over DoH (outside the VPN tunnel). Firefox defaults to Cloudflare DoH. Either way, neither goes through your VPN by default.\nTo prevent leaks, disable DoH in your browser entirely, or point it to your VPN provider\u0026rsquo;s DoH endpoint. Mullvad runs dns.mullvad.net, which is ad-blocking, no-logging, and available over DoH/DoT.\nTest externally at dnsleaktest.com. If your ISP\u0026rsquo;s servers appear in the results, you have a leak, and you need to fix it.\nThe diagram below shows the 2 paths DNS queries can take:\nExpand image How Networks Know You\u0026rsquo;re Using a VPN IT teams use Deep Packet Inspection (DPI) firewalls that inspect packets and analyze the content, structure, and timing of traffic to fingerprint protocols.\nThe problem here is that WireGuard has an easy to identify fingerprint. The WireGuard handshake initiation message is exactly 148 bytes and the response is 92 bytes. These are constants defined in the WireGuard spec. Beyond the handshake, WireGuard traffic has consistent patterns: it\u0026rsquo;s always UDP, keepalive intervals happen every 25 seconds by default, and the payload distribution has a particular structure. Any DPI system with a WireGuard rule can identify this fingerprint in an instant and flag it.\nThis matters in 3 scenarios:\nCorporate Networks With Next-Gen Firewalls: Palo Alto and similar DPI firewalls have WireGuard signatures in their rulesets. An IT team can easily block WireGuard (or any VPN protocol) with a single rule. Restrictive Countries: Countries with network restrictions rely on DPI systems to identify and block VPN traffic. Mullvad itself notes that WireGuard is blocked in China. ISP Throttling: Some ISPs throttle VPN traffic based on protocol identification without fully blocking it. For example, your VPN works but slows down past a certain data threshold. Expand image Hiding the Fact You\u0026rsquo;re Using a VPN Obfuscation is not built into WireGuard so you must add an obfuscation layer on top.\nFor example, Mullvad provides these obfuscation (anti-censorship) methods for bypassing DPI restrictions:\nShadowsocks wraps WireGuard traffic inside an obfuscated proxy stream. The resulting traffic looks like random encrypted bytes with no identifiable protocol signature (no WireGuard handshake headers, no consistent port, no UDP-only behavior). To a DPI system, it\u0026rsquo;s indistinguishable from TLS or any other encrypted stream. udp2tcp wraps WireGuard UDP packets inside a TCP stream. Useful on networks that block UDP entirely. The tradeoff for this is performance as you lose UDP\u0026rsquo;s speed. Use it when you need it, not as a default. QUIC tunnels WireGuard inside QUIC frames (another obfuscation transport that\u0026rsquo;s harder to fingerprint than raw UDP and harder to block than TCP-based transports). LWO (Lightweight Obfuscation) is the lowest-overhead option with minimal packet transformation, no proxy layer. You can use this for environments where plain WireGuard gets blocked but full Shadowsocks feels like overkill. WireGuard port switching is the lightest of all. Some networks block the default WireGuard port (51820/UDP) but leave other UDP ports open. wireguard-port lets you switch to a different port without any obfuscation overhead (useful when the problem is a simple port block rather than actual DPI). auto mode lets the app try methods in order of overhead until one connects. Useful if you don\u0026rsquo;t know exactly what the network is blocking and don\u0026rsquo;t want to guess manually. # Check current anti-censorship status mullvad status # Full list of available modes: auto, off, wireguard-port, udp2tcp, shadowsocks, quic, lwo mullvad anti-censorship set mode auto # let the app figure it out mullvad anti-censorship set mode shadowsocks # best DPI evasion mullvad anti-censorship set mode udp2tcp # for networks blocking UDP mullvad anti-censorship set mode quic # WireGuard over QUIC/HTTP3 mullvad anti-censorship set mode lwo # lightweight header scrambling mullvad anti-censorship set mode off # back to plain WireGuard # Set the WireGuard port to a different one mullvad anti-censorship set wireguard-port 4440 # Each method has its own subcommand for advanced per-method config mullvad anti-censorship set shadowsocks --help mullvad anti-censorship set udp2tcp --helpMullvad also has a feature called DAITA (Defense Against AI-guided Traffic Analysis), which rather than bypassing DPI, it combats traffic analysis using machine learning. It adds random dummy data to the VPN traffic to prevent systems from recognizing VPN traffic from correlating packet timing, size, and patterns.\nWhat this looks like using an shadowsocks obfuscation method in a censorship network:\nWhat\u0026rsquo;s Next? This is a good stopping point for the next part which will be more hands-on for different use cases. Planning to do some specific configurations and devices so it will take more time.\nPrevious Parts:\nPart 1: Understanding VPNs Part 1. Part 2: Understanding VPNs Part 2. ","permalink":"https://danoss.me/en/categories/security/understanding-vpns-part-3/","summary":"Part 3 of a multi-part writing about VPNs","title":"Understanding VPNs Part 3"},{"content":"Part 2 of a 3-part series on VPNs.\nThis part focuses more in depth on VPNs. I\u0026rsquo;ll use my actual Mullvad VPN setup on Fedora Linux to go through what happens when you connect, how the kernel handles traffic, etc.\nThis post is not an ad of Mullvad, but just a writing from a happy user.\nWhat Happens When You Connect Here\u0026rsquo;s the flow of a normal internet connection vs. one going through a VPN:\nNotice that with a VPN, traffic hits the wg0 interface first, gets encrypted, then goes to the real network interface as an encrypted packet. The ISP only sees the outer packet. Everything inside is encrypted.\nNow let\u0026rsquo;s set this up.\nInstalling Mullvad on Linux I use Fedora, but this works on any Linux distro. There are two ways to run Mullvad: using the official app (from https://mullvad.net/) (which handles everything for you), or using raw WireGuard tools (which gives you full control).\nFedora has WireGuard built directly into the kernel, so no third-party dependency is required.\nThe Official Mullvad App To install the official Mullvad app in Fedora Linux follow the steps of: https://mullvad.net/en/help/install-mullvad-app-linux#fedora.\n# Add the official Mullvad repository sudo dnf config-manager addrepo --from-repofile=https://repository.mullvad.net/rpm/stable/mullvad.repo # Install the official app sudo dnf install mullvad-vpnAfter installation, log in and connect using the CLI (you can also do this with the GUI).\n# Log in to your Mullvad account ID (you\u0026#39;ll get this when you register) mullvad account login 123456789 # Check account information (account number, expiration, device name) mullvad account get # Set your preferred country (Sweden in this example) mullvad relay set location se # Connect mullvad connect # Check status (relay, features, location) mullvad statusThe official app creates the WireGuard interface, configures routing, sets up the kill switch, handles DNS, everything. Really useful.\nWhat the App Creates When Mullvad connects, it creates a WireGuard interface. You can see it like this:\nip link show type wireguard # Output 42: wg0-mullvad: \u0026lt;POINTOPOINT,UP,LOWER_UP\u0026gt; mtu 1380 qdisc noqueue state UNKNOWNSame as above but view at the specific interface.\nip addr show wg0-mullvad # Output 42: wg0-mullvad: \u0026lt;POINTOPOINT,UP,LOWER_UP\u0026gt; mtu 1380 qdisc noqueue state UNKNOWNYou can also check the WireGuard configuration (you may need to install the wireguard-tools package if you don\u0026rsquo;t have it).\nsudo wg show # Output interface: wg0-mullvad public key: \u0026lt;your public key\u0026gt; private key: (hidden) listening port: 42069 fwmark: 0xca6c peer: \u0026lt;Mullvad server public key\u0026gt; preshared key: (hidden) endpoint: (ip:port) allowed ips: 0.0.0.0/0, ::/0 latest handshake: 8 seconds ago transfer: 145.67 MiB received, 12.34 MiB sentSome things to note in the output:\nallowed ips: 0.0.0.0/0, ::/0: This means ALL traffic (IPv4 and IPv6) should go through this peer. It\u0026rsquo;s a full tunnel. latest handshake: Should be recent. If this is blank or old, the tunnel isn\u0026rsquo;t working. Virtual Network Interfaces (wg0-mullvad) Mullvad automatically creates the wg0-mullvad virtual network interface for the VPN.\nOn Linux, there are 2 main types of virtual interfaces used by VPNs:\nTUN (Layer 3): Handles raw IP packets. No Ethernet headers. This is what OpenVPN uses. The kernel sends an IP packet to the TUN interface, and a userspace program (OpenVPN) reads it from a file descriptor at /dev/net/tun, encrypts it, and sends it out a regular socket.\nWireGuard (Kernel Module): WireGuard doesn\u0026rsquo;t use TUN. It registers its own interface type directly in the kernel. When you create a WireGuard interface, the kernel module itself handles encryption. No userspace involvement.\nThis distinction matters for performance, as every context switch between kernel and userspace costs CPU cycles. That\u0026rsquo;s why WireGuard is faster than OpenVPN.\nYou can verify the WireGuard kernel module is loaded.\nlsmod | grep wireguard # Output wireguard 126976 0 libcurve25519 65536 1 wireguard ip6_udp_tunnel 16384 2 wireguard,vxlan udp_tunnel 40960 2 wireguard,vxlanTracing a Packet From Your Browser to the Wire Here\u0026rsquo;s what happens when you visit a website while connected to Mullvad.\nSay your browser requests example.com (93.184.216.34).\nThe Application Sends Data\nYour browser calls send(). The kernel\u0026rsquo;s TCP/IP stack creates a packet:\nSource IP: 10.66.X.X (your tunnel IP) Destination IP: 93.184.216.34 The Kernel Consults The Routing Table\nMullvad uses policy routing, so the VPN default route lives in a separate table (not the main one). The kernel checks the policy rules first.\n# Show Policy Rules # Policy rules divert unmarked traffic to the VPN routing table ip rule show # Output 0: from all lookup local 32765: not from all fwmark 0x6d6f6c65 lookup 1836018789 32766: from all lookup main 32767: from all lookup default# Show VPN Routing Table # The VPN routing table sends everything through the tunnel ip route show table 1836018789 # table value from previous output # Output default dev wg0-mullvad scope link# SHow Main Routing Table # The main table still has your real gateway (for VPN server traffic) $ ip route show # Output default via 192.168.1.1 dev wlan0 10.66.0.0/16 dev wg0-mullvad proto kernel scope link src 10.66.X.X 192.168.1.0/24 dev wlan0 proto kernel scope link src 192.168.1.100Since our packet to 93.184.216.34 (example.com) has no 0x6d6f6c65 mark, it matches rule 32765 so it gets routed through VPN table 1836018789 → wg0-mullvad interface, where WireGuard encrypts it and stamps it with the mark.\nThat stamp is the kernel\u0026rsquo;s way of saying \u0026ldquo;this packet has already been processed.\u0026rdquo; On its second pass through the policy rules, the mark causes rule 32765 to skip it, letting it fall through to the main table and out through the real gateway, this time headed to the Mullvad server, not example.com.\nYour ISP sees nothing but encrypted UDP traffic destined for a Mullvad IP.\nDiagram to Show this Process More Easily\nExpand image How Mullvad Handles Routing (fwmarks) A routing loop is what happens when a packet keeps getting sent through the VPN tunnel indefinitely, the tunnel tries to send traffic through itself, which creates another packet that needs to go through the tunnel, which creates another, and so on. Mullvad solves this with firewall marks (fwmark) and policy routing, which is cleaner than the naive fix of adding a static route for each VPN server IP.\nMullvad sets fwmark: 0xca6c on the WireGuard interface (wg0-mullvad), but the routing logic relies on a separate mark (0x6d6f6c65) applied to packets that are already encrypted and destined for the VPN server.\n# Policy routing rules ip rule show # Output 0: from all lookup local 32765: not from all fwmark 0x6d6f6c65 lookup 1836018789 32766: from all lookup main 32767: from all lookup default# What lives in the VPN routing table ip route show table 1836018789 # Output default dev wg0-mullvad scope linkThe VPN\u0026rsquo;s default route lives in a separate table (1836018789), not in the main routing table. The main table keeps your original gateway untouched. The policy rule is what decides which table a packet uses:\nYour app sends a packet with no mark Rule 32765 matches: \u0026ldquo;no 0x6d6f6c65 mark? → use VPN table 1836018789\u0026rdquo; VPN table 1836018789 sends it through interface wg0-mullvad, where it gets encrypted and stamped with 0x6d6f6c65 The encrypted packet hits the rules again, this time rule 32765 skips it, it falls through to the main table, and exits through wlan0 to your ISP headed for the Mullvad server. This prevents routing look with no hardcoded server IPs. If Mullvad changes servers or you switch locations, nothing breaks, the logic is based on packet marks, not destinations.\nThis same mechanism is what makes split tunneling possible. To bypass the VPN for a specific app, Mullvad marks its traffic with 0x00000f41 (for connection tracking) and 0x6d6f6c65 (for routing), which lets it skip the VPN table and go out through the normal interface. This is exactly how running Mullvad and Tailscale simultaneously works On Mullvad VPN and Linux\u0026hellip; and Tailscale, nftables rules mark Tailscale traffic with those same marks, so it routes outside the tunnel while everything else stays encrypted.\nMore information split tunneling: https://mullvad.net/en/help/split-tunneling-with-the-mullvad-app\nThe Kill Switch Feature A kill switch prevents traffic from leaking if the VPN tunnel goes down. When the Mullvad kill switch is active, Mullvad creates nftables (or iptables) rules that drop all traffic not going through the VPN interface.\nCheck Mullvad\u0026rsquo;s kill switch status.\nmullvad lockdown-mode get # Output Block traffic when the VPN is disconnected: onEnable Mullvad\u0026rsquo;s kill switch.\nmullvad lockdown-mode set onWith this feature, if the WireGuard tunnel drops, there\u0026rsquo;s no wg0-mullvad interface anymore. All app traffic hits the \u0026ldquo;DROP everything else\u0026rdquo; rule. Nothing leaks. No DNS queries go to your ISP. No application traffic goes out unencrypted. Nothing.\nThe Next Part For part 3, the plan is to go more in depth, DNS leaks in VPNs, techniques to avoid evading deep packet inspection, specially for environments where VPN connections are actively detected and blocked. Corporate networks, firewalls, etc.\n","permalink":"https://danoss.me/en/categories/security/understanding-vpns-part-2/","summary":"Part 2 of a multi-part writing about VPNs (focusing in Mullvad VPN)","title":"Understanding VPNs Part 2"},{"content":"What are VPNs?\nA VPN (Virtual Private Network) is a must-use tool in today\u0026rsquo;s world, you either use or accept that your ISP, government, and every ad company on the world knows exactly what you do online.\nThis is Part 1 of a multi-part writing about VPNs. This one covers the basics: what is a VPN, how it protects you, what makes a good VPN provider, and why free VPNs are worse than using nothing at all.\nimage source: https://howtofix.guide/vpn/.\nWhy Privacy Matters Internet privacy has been under attack for many years now. Governments want to know what you\u0026rsquo;re doing online to prevent protesters from rising up. ISPs (Internet Service Providers) want to sell and share what you\u0026rsquo;re doing online to make money. Advertisers want to profile what you\u0026rsquo;re doing online to show you relevant ads. And data brokers want to package all of the above and sell it to whoever pays.\nA VPN won\u0026rsquo;t solve every privacy problem (nothing will), but it plugs a massive hole. It removes your ISP from spying on you by encrypting your Internet connection and routing it through a server you choose (like a forward proxy). Instead of your ISP seeing that you visited example.com, they see encrypted traffic going to a VPN server IP address. That\u0026rsquo;s it. They know you\u0026rsquo;re using a VPN, but they can\u0026rsquo;t see what\u0026rsquo;s inside the tunnel (the VPN provide can though, so choose wisely).\nWhat a VPN Actually Does A VPN creates an encrypted tunnel between your device and a VPN server. All of your internet traffic passes through this tunnel before reaching the public Internet.\nWhen you connect to a VPN, several things happen. First, your VPN client in your device (computer, phone, etc.) establishes an encrypted connection to a VPN server using a protocol like WireGuard or OpenVPN. This involves a cryptographic handshake where your device and the server authenticate each other and agree on session encryption keys.\nOnce the tunnel is up, your device creates a virtual network interface and all outgoing traffic gets routed through this interface. The VPN client encrypts each packet, wraps it in a new UDP (or TCP) packet addressed to the VPN server, and sends it out through your real network interface. This is called encapsulation, you\u0026rsquo;re putting an encrypted packet inside a regular packet.\nOn the server side, the reverse happens. The VPN server receives the outer packet, strips the encapsulation, decrypts the inner packet, and forwards it to the destination on the public internet. The response follows the same path back: the destination server sends its response to the VPN server, the VPN server encrypts it, sends it through the tunnel to your device, and your device decrypts it.\nSo a VPN provides these benefits:\nYour IP Address is Hidden: Websites and services you connect to see the VPN server\u0026rsquo;s IP address, not yours. This means they can\u0026rsquo;t determine your physical location or tie your activity to your home network. Your ISP Cannot See your Traffic: Your ISP can only see encrypted packets going to a VPN server IP. They don\u0026rsquo;t know what websites you\u0026rsquo;re visiting, what you\u0026rsquo;re downloading, or what services you\u0026rsquo;re using. DNS Queries go Through the Tunnel: A properly configured VPN routes your DNS lookups through the encrypted tunnel and resolves them on the VPN provider\u0026rsquo;s DNS servers (or a DNS server of your choosing). This prevents DNS leaks, which are one of the most common ways that VPN users accidentally expose their browsing activity. Without DNS leak protection, your browser might still send DNS queries to your ISP\u0026rsquo;s resolver even though your actual traffic goes through the VPN. All Traffic is Encrypted in Transit: Even if someone intercepts the packets between your device and the VPN server (for example, on a public Wi-Fi network), they see nothing but encrypted data. View Content From Any Location (Bonus): A bonus benefit is that you can connect to a VPN server in a different country and view content that is blocked in your own country. Types of VPN Protocols A VPN protocol is the set of rules that determines how your data is encrypted and transmitted between your device and the VPN server. The 2 main VPN protocols today are WireGuard and OpenVPN.\nOpenVPN The OG protocol.\nOpenVPN is fully configurable, you can choose your cipher (AES-256-GCM, AES-256-CBC), your key exchange method (RSA 2048/4096, ECDH), your authentication algorithm (SHA-256, SHA-512), and your transport layer (UDP, TCP).\nThis means OpenVPN can be configured for very specific scenarios, which is useful in some enterprise environment and environment complex censorship infrastructure.\nThe key exchange process in OpenVPN works through TLS handshakes (the same mechanism used by HTTPS). When your client connects to a server, they perform a TLS handshake using X.509 certificates and/or pre-shared keys to authenticate each other and establish a session key. This session key is then used for the symmetric encryption of your actual traffic.\nThis means you can configure OpenVPN to route traffic over TCP on port 443, which is really useful in restrictive network environments because it\u0026rsquo;s essentially indistinguishable from regular HTTPS traffic at the connection level. A deep packet inspector can tell the difference if they analyze the TLS fingerprint and traffic patterns, but basic firewalls that simply allow port 443 traffic will let it through. This makes OpenVPN a useful in censored environments and environment with complex security restrictions.\nThe downside is that it has a large codebase, which means a larger attack surface and overhead, making the connections slower than WireGuard.\nWireGuard WireGuard is today\u0026rsquo;s standard and the go-to option for most use cases.\nInstead of allowing you to configure your VPN connections with as many options as you want, you get a pre-selection of the best modern options and that\u0026rsquo;s it.\nDuring a WireGuard handshake, the initiator sends a handshake initiation message with its ephemeral public key and its static public key (encrypted). The responder verifies the initiator\u0026rsquo;s identity using the pre-configured public key, generates its own ephemeral key pair, performs 4 Diffie-Hellman calculations using combinations of the static and ephemeral keys, and sends back a handshake response. The both sides now derive identical symmetric session keys from these shared secrets.\nWireGuard also supports an optional pre-shared symmetric key (PSK) that can be configured per-peer. This provides an additional layer of symmetric encryption on top of the asymmetric cryptography, which is specifically designed to protect against potential future quantum computing attacks on Curve25519.\nThe handshake automatically re-keys every few minutes to maintain perfect forward secrecy. If an attacker somehow compromises a session key, they can only decrypt traffic from that specific session, not past or future sessions.\nAlso, the WireGuard codebase is much smaller than OpenVPN, which means less overhead and exposure.\nAlso important, WireGuard was originally designed with static IP assignments per peer, meaning the server needs to know which public key maps to which tunnel IP. Some VPN providers (for example, NordVPN with their NordLynx implementation) have built systems on top of WireGuard to address this, using double NAT or ephemeral key rotation so that the server doesn\u0026rsquo;t persistently associate your public key with your activity. Mullvad and Proton both handle this transparently in their implementations.\nWireGuard only operates over UDP, which is great for speed but can be a problem on networks that restrict UDP traffic. It also doesn\u0026rsquo;t have built-in traffic obfuscation, meaning deep packet inspection can identify it as a VPN connection based on the packet structure.\nWhich Should You Use? For most users, WireGuard is the better choice, it is faster and has a smaller attack surface. Both Mullvad and Proton VPN support it as its default.\nSince OpenVPN is much more customizable, it works great in environments with restrictive firewalls, networks that block UDP traffic, or censored Internet environments where you need to hide inside TCP port 443 traffic.\nProton VPN supports both protocols while Mullvad went WireGuard-only, but added Shadowsocks obfuscation and UDP-over-TCP to compensate for the censorship-bypass scenarios where OpenVPN shines.\nUsing a VPN on Windows vs Linux You might think the underlying OS doesn\u0026rsquo;t matter as much, you are wrong. I won\u0026rsquo;t be talking about MacOS/Apple because their products are garbage, hate being locked in an ecosystem.\nWindows and Telemetry Windows collects a lot of telemetry data by default and most of it is \u0026ldquo;Required\u0026rdquo; by Microsoft (why would someone willingly choose Windows as its main OS is beyond me).\nSome of the required telemetry in Windows includes device specifications, hardware config, OS version, driver information, system crash reports, and Windows Update status. \u0026ldquo;Optional\u0026rdquo; adds app usage patterns, browsing history from Edge, inking and typing patterns, and enhanced error reporting. Your typing patterns\u0026hellip; hmm...\nThese telemetry endpoints are hardcoded in the OS and while you can block them at the firewall or DNS level, Windows updates usually re-enable telemetry settings and add new endpoints.\nSo you can have a VPN running, encrypting all your internet traffic, and Windows is still phoning home to Microsoft through that same encrypted tunnel. Your ISP can\u0026rsquo;t see the telemetry data because it\u0026rsquo;s going through the VPN, but Microsoft can (because they\u0026rsquo;re the destination). You moved the surveillance from your ISP to Microsoft. Progress I guess?\nLinux and Control Linux takes the opposite approach. Most distributions collect zero telemetry by default. No background service phoning home. No usage analytics. No mandatory cloud integration. The OS doesn\u0026rsquo;t make network connections you didn\u0026rsquo;t explicitly configure (you can opt-in of course).\nOn a Linux system, you have full transparency over every network connection. You can run ss -tunap or netstat to see exactly what\u0026rsquo;s communicating and to where and see which services are running with systemctl. This means that on a Linux machine with a VPN running, the only traffic leaving your system is traffic you knowingly generated.\nOn Linux, WireGuard runs as a kernel module (wireguard.ko), meaning the encryption and decryption happen in kernel space with minimal context switching overhead. The virtual interface (wg0) is a first-class kernel network interface.\nYou can also control more in depth how the VPN integrates with your system:\nnftables / iptables for Firewall Rules: You can build a kill switch that drops all traffic if the VPN tunnel goes down, ensure that only VPN traffic leaves your machine, or route specific applications through or around the VPN tunnel. On my setup, I use nftables rules to run both Mullvad and Tailscale simultaneously (more details here). Network Namespaces for Application Isolation: Linux network namespaces let you create entirely separate networking environments per process or group of processes (cgroups). You can put a specific application in a namespace that only has access to the VPN interface, making it physically impossible for that application to leak traffic outside the tunnel. resolvectl / systemd-resolved for DNS Control: You can configure per-interface DNS settings, ensuring that DNS queries always go through the VPN\u0026rsquo;s DNS servers and never leak to your ISP\u0026rsquo;s resolver. On Fedora and most modern distributions, systemd-resolved handles this cleanly when the VPN connection is configured correctly. In the End It DOES matter\u0026hellip;\nA VPN on Windows still protects your traffic from your ISP, but if you care about privacy, use a VPN on Linux. On Linux, you know exactly what traffic is leaving your machine and where it\u0026rsquo;s going.\nZero-Knowledge VPN Providers Most VPN providers are garbage, they are usually honeypots that get your data and sell it, some throw around the concept of \u0026ldquo;no-log VPN\u0026rdquo; but have no evidence or audits to back it up.\nWhat you want is a zero-knowledge VPN provider: one that is technically and operationally designed so that even if their servers were seized or they were compelled by law enforcement, they would have nothing useful to hand over.\nA truly zero-knowledge VPN service has:\nNo logging of user activity, traffic, connection timestamps, or metadata. No requirement for personal information during signup (no email, no name). Anonymous payment options (cryptocurrency like Monero, cash). Regular independent security audits that verify these claims. Open-source clients so the code can be independently reviewed. The are more VPNs but I have only experience with Mullvad and Proton VPN, so I\u0026rsquo;ll talk about them below.\nProton VPN Proton VPN is really good and has a solid track record.\nProton is based in Switzerland, a country with strong privacy laws, however the country is currently considering new surveillance legislation (the OSCPT ordinance) that could impact privacy. The \u0026ldquo;good\u0026rdquo; thing is that Proton is already moving some infrastructure to the EU as a precaution, but the whole EU is moving to an anti-privacy stance, so it might completely worthless.\nNo-Logs Policy\nProton VPN maintains a no-logs policy that has been independently verified. They\u0026rsquo;ve now passed 4th consecutive annual third-party audits, which is a good thing.\nLocal Infrastructure\nProton runs their VPN on bare-metal servers they fully own and control. There is no 3rd-party cloud provider or middle man that could get user data from the physical hardware.\nSecurity Audits\nProton passed a SOC 2 Type II audit in 2025, which verifies the proper implementation of security controls across their infrastructure. This is a different kind of audit from the no-logs verification; it validates their overall security posture and operational controls.\nThe Interesting Part\nProton is secure and all, but they have a pattern of handing over user data to authorities when compelled by Swiss courts, and it has led to real people being identified and arrested.\nIn 2021, Proton Mail provided the IP address of a French climate activist to Swiss authorities, who shared it with French police. In 2024, they handed over a recovery email address that Spanish police used to identify a Catalan independence activist. And just this week, 404 Media reported that Proton Mail provided payment data that the FBI used to unmask an anonymous Stop Cop City protester in Atlanta.\nProton\u0026rsquo;s defense is always the same: they comply with Swiss court orders, they can\u0026rsquo;t decrypt email content, and they only hand over what limited metadata they have. They also emphasize that their VPN service, unlike Proton Mail, doesn\u0026rsquo;t log IP addresses, so the VPN side hasn\u0026rsquo;t been implicated in these cases. That distinction matters, but makes you not trust them 100% (at least in my case, specially after hearing the CEO opinions on some matters).\nSome key takeaways from these is to not pay Proton with a credit card, and use a fake recovery email, do your opsec homework.\nMullvad VPN This is another good option for a zero-knowledge VPN with pros and cons.\nMullvad is based in Sweden which also has strong privacy protection laws (but again, as all EU, they are moving to an anti-privacy state).\nSignup Process\nThe signup process is incredible and tells you everything you need to know about their philosophy. No email. No name. No nothing. You click \u0026ldquo;Generate account number\u0026rdquo; and get a randomly assigned 16-digit number. That\u0026rsquo;s your ID. Done.\nAnonymous Payment\nFor payment, you can use credit card or PayPal (less private), cryptocurrency like Bitcoin and Monero (more private), or you can literally put cash in an envelope with your account number and mail it to their office in Sweden. They\u0026rsquo;ll open the envelope, add time to your account, and shred the paper. This level of anonymous access is unmatched, love it.\nNo-Logs Policy\nMullvad has been independently audited multiple times by security firms and found zero critical, high, or medium-severity issues, only a single low-severity input validation weakness that was promptly fixed. You can see the audit information in their website.\nA good evidence of this came in 2023, when Swedish police physically showed up at Mullvad\u0026rsquo;s office with a search warrant, intending to seize computers with customer data. Mullvad\u0026rsquo;s staff demonstrated how their service works and that no customer data existed. After consulting the prosecutor, the police left without taking anything.\nThe Interesting Part\nSome people claim Mullvad is a government honeypot, designed to attract privacy-conscious users, which could be true\u0026hellip; or not?\nTheir service is very good but I can\u0026rsquo;t prove this isn\u0026rsquo;t the case. I trust the independent audits, open-source, anonymous sign-up, and anonymous payment methods provided as a good sign on a privacy-first company. I could be wrong though, its just a matter of trust.\nThe Dangers of Free VPNs Free VPNs are (usually) a privacy nightmare, worse than no VPN at all if I\u0026rsquo;m honest.\nRunning a VPN service costs money. If a VPN is not charging you, they are making money by selling your data.\nSome recommended reads about the topic:\nAre free VPNs safe? Here\u0026rsquo;s what the research says VPN Transparency Report 2025 60% of free VPNs could be selling your data by 2025 What a VPN Doesn\u0026rsquo;t Protect You From Some of VPN marketing claims like \u0026ldquo;complete anonymity\u0026rdquo; is bullshit, here\u0026rsquo;s the truth:\nA VPN Doesn\u0026rsquo;t Make You Anonymous: If you log into Google while connected to a VPN, Google still knows exactly who you are. You typed in your username and password. The VPN hides your IP address from the destination, but it doesn\u0026rsquo;t change your login session, your browser fingerprint, etc. If you\u0026rsquo;re using a VPN to \u0026ldquo;hide from Google\u0026rdquo; while logged into Gmail, I have bad news for you. A VPN Doesn\u0026rsquo;t Protect Against Browser Fingerprinting: Your browser reveals a lot of identifying information about you: screen resolution, installed fonts, GPU renderer, timezone, language settings, installed plugins, canvas rendering, WebGL, etc. This combination is often unique enough to identify you without ever needing your IP address. A VPN doesn\u0026rsquo;t change any of this, you will still be identified. More details on fingerprinting here. A VPN Doesn\u0026rsquo;t Guarantee That Your VPN Provider Isn\u0026rsquo;t Logging Your Data: By using a VPN, you\u0026rsquo;re shifting trust from your ISP to your VPN provider. Your VPN provider can theoretically see all your traffic, however, a good VPN provider has a verified no-logs policy and audited infrastructure to protect you. Your ISP has none of these. So you\u0026rsquo;re trading a guaranteed spy for a hopefully-trustworthy middleman. Choose wisely. A Quick Summary Get a VPN. Use it. Pay for it with Monero or similar privacy-first cryptocurrency.\nChoose a reliable VPN provider that has been audited, is open-source, has anonymous payment options, and a good track record of protecting user data from law enforcement. Mullvad and Proton VPN are the two I recommend but there are more, do your own research.\nAvoid free VPNs. They are not free. You\u0026rsquo;re paying with your data, which is sold to who knows who.\nVPNs don\u0026rsquo;t provide complete anonymity. Combine it with a privacy-focused browser (Firefox-or forks with hardened settings for example - more details here), a content blocker (uBlock Origin), encrypted DNS, etc. There is no perfect privacy but some is better than nothing.\nPart 2 will be more in depth VPN information and part 3 I\u0026rsquo;ll do more research on how to prevent VPN traffic from being detected by corporate networks, governments, and so on.\n","permalink":"https://danoss.me/en/categories/security/understanding-vpns-part-1/","summary":"Part 1 of a multi-part writing about VPNs","title":"Understanding VPNs Part 1"},{"content":"Most of the world uses Google Chrome as its main browser for many reasons: It\u0026rsquo;s user-friendly, linked to your Google account, comes pre-installed in many devices, and so on.\nNow, looking at it from a privacy perspective, you are living in a nightmare. Going into the rabbit hole of privacy and security makes this clearer.\nBuilt By an Advertising Company? Google is pretty much an advertising company, lots of their revenue comes from ads. I would even say that Chrome is not a product but a pipeline to feed your data into Google for targeted advertising. Google knows everything you are doing online.\nFor example, in 2018, cryptographer Matthew Green wrote the blog post \u0026ldquo;Why I\u0026rsquo;m Done With Chrome\u0026rdquo;, which talks about a change in Chrome sign-in experience. Previously, you could use Chrome without logging in. Then, Google decided that simply logging into Gmail would automatically sign you into the browser without your consent and start syncing your browsing data to Google\u0026rsquo;s servers.\nThis is not just a change, it a behavior, Google will change how their products work to invade your privacy.\nWhat Chrome Actually Collects When you sign-in to your Google account in Chrome, the browser can correlate your browsing history, search queries, location, and cookie information across devices and sessions to feed their advertising engine. So, if you use Google products in the Chrome browser, Google has a full picture of your life.\nEven if you are using Incognito Mode, you are not anonymous. Google will still see your traffic, log DNS lookups, and track your online activity. This is so serious that Google actually settled a $5 billion class action lawsuit from users who claimed they were misled about what Incognito Mode actually does, Google agreed to delete billions of records it had collected from users it had told were browsing privately (source).\nThird-Party Cookies and Privacy Sandbox Third-party cookies allow advertisers to follow you around the web from site to site to create a digital profile of you and sell your data. Firefox, Safari, and most other browsers block these third-party cookies by default to improve user privacy, not Google though.\nIn 2020 Google announced they\u0026rsquo;d be phasing out third-party cookies in Chrome (source), however they decided to do a complete U-turn and abandon this plan (source).\nInstead, Google introduced the Privacy Sandbox, in which rather than letting third-party trackers follow you around the web, Privacy Sandbox moves the tracking into Chrome itself. Your browser watches what you do, builds a profile of your interests, and then shares it with advertisers. Still a privacy nightmare.\nThen in 2025, Google announced Privacy Sandbox was dead (source) and will be keeping third-party cookies in Chrome indefinitely.\nGoogle Killed the Best Ad Blocker in Chrome In 2025, Google fully transitioned from Manifest V2 to Manifest V3, a new framework on how Chrome extensions work. The main change was replacing the webRequest API, which extensions like uBlock Origin (the most effective content blocker ever built) used to intercept network requests in real time, with the far more limited declarativeNetRequest API (source).\nThis resulted in the full version of uBlock Origin to no longer work in Chrome. A Lite version exists that\u0026rsquo;s been rebuilt around Manifest V3\u0026rsquo;s constraints, but it\u0026rsquo;s less capable than the full version.\nSo Google, an advertising company, redesigned the extension platform in a way that specifically cripples the best ad-blocking software, which conveniently benefits their core business. We might be onto something here.\nChrome vs Chromium Chrome is Google\u0026rsquo;s proprietary browser built on top of Chromium, which is open source.\nChromium, in a sense, is more trustworthy than Chrome. So, some people prefer to use Chromium-based privacy-focused browsers like Brave instead of Firefox, which is a solid option.\nSo Should You Switch? Yes, post over\u0026hellip; or\u0026hellip;\nBefore changing browsers, even if Chrome is terrible for privacy, it does have excellent extensions that allow you to easily integrate third-party tools and automate browser workflows, and many AI-powered browser extensions are Chrome-first (or Chrome-only).\nTake that into consideration before fully migrating. If you prefer some of this functionality I would recommend to keep Chrome as a secondary browser to use when needed but not your main if you value privacy.\nThat said, Firefox isn\u0026rsquo;t perfect and Mozilla fumbles the bag as well. In 2025, Mozilla quietly updated their Terms of Use with broad licensing language that appeared to grant Mozilla rights to whatever data you input into the browser. They walked it back after massive backlash, claiming it was just poorly a worded update, but we are not that dumb (source).\nThen in late 2025, the new Mozilla CEO announced Firefox would evolve into a \u0026ldquo;modern AI browser\u0026rdquo;, raising question on where this AI-gathered data lives, privacy, and security of the feature (source).\nFirefox and Its Privacy-Respecting Forks Even if Mozilla has its quirks and concerns, they are a much better option for privacy-focused individuals.\nFirefox blocks third-party cookies by default through Enhanced Tracking Protection, supports the full version of uBlock Origin, and supports about:config, where you can tune hundreds of settings that Chrome doesn\u0026rsquo;t expose.\nPersonally, instead of just vanilla Firefox, i prefer to use a privacy-focused Firefox fork, which works well and is more locked down.\nLibreWolf is the most-privacy focused fork with great defaults out of the box. The only trade-off i see is that some sites break because of the aggressive fingerprinting resistance and that there are a low number of maintainers. Waterfox is less aggressive than LibreWolf in terms of hardening but more focused on removing the telemetry and \u0026ldquo;phone home\u0026rdquo; behavior Mozilla has added over the years while keeping Firefox\u0026rsquo;s extension compatibility and user experience mostly intact. This is a good middle-ground option for most users. In the end, its personal preference on what you value from user-experience and privacy. You can choose to remain in Chrome, switch to a Chromium-based browser like Brave, go with Firefox and forks, or another browser altogether, choose wisely.\nRelated Posts What is Browser Fingerprinting? Your Digital Footprint ","permalink":"https://danoss.me/en/categories/others/stop-using-chrome/","summary":"Chrome isn\u0026rsquo;t a browser, it\u0026rsquo;s Google\u0026rsquo;s data collection pipeline with a UI","title":"Stop Using Chrome: A Deep Dive Into Google Chrome Privacy Problem"},{"content":"SSL/TLS certificates are not that bad once you understand them, unlike DNS (which is always painful) getting HTTPS working for self-hosted services is actually pretty simple.\nMost of my home server services run as Podman containers accessible through a VPS gateway with a public IP. To access those services securely from anywhere, they all need to be behind HTTPS, which means i have to use certificates.\nSSL/TLS Certificates When you see a padlock icon in your web browser, a TLS certificate is doing 2 things: encrypting your connection so nobody can read the traffic in transit, and verifying that the server you\u0026rsquo;re talking to is actually who it claims to be.\nSSL (Secure Sockets Layer) is the old, deprecated name of the protocol that everyone still uses out of habit (and feels better to say honestly). TLS (Transport Layer Security) is the actual current protocol. Both names are usually used interchangeably, but today TLS is the go to.\nFor self-hosted environments you have 2 options. Using a public Certificate Authority like Let\u0026rsquo;s Encrypt, which is free, automated, trusted by every browser or run your own private CA for internal services, which gives you more control but is more complex to setup.\nFor self-hosted services, the go-to Certificate Authority is Let\u0026rsquo;s Encrypt, which is free, automated, and trusted by every browser out of the box. The other option is running your own private CA, which gives you more control but is more complex to set up and maintain.\nMy Setup I\u0026rsquo;m a believer in KISS (Keep It Simple, Stupid). The topology looks like this:\nInternet → VPS (Headscale + port forwarding) → Tailscale VPN → Home Server (Caddy + Podman containers)\nCloudflare manages DNS, pointing each subdomain to the VPS public IP. The VPS runs Headscale for VPN coordination and forwards incoming traffic on port 443 through the Tailscale tunnel to the home server. Caddy runs on the home server, handles TLS termination, and reverse proxies requests to the right container. Services get their own subdomains, like auth.example.com, home.example.com, and so on.\nSince the home server isn\u0026rsquo;t directly reachable from the internet, Caddy can\u0026rsquo;t use HTTP-01 challenges for Let\u0026rsquo;s Encrypt, those require Let\u0026rsquo;s Encrypt to connect back to your server on port 80. Instead, Caddy uses DNS-01 challenges via the Cloudflare API, where domain ownership is verified through a DNS TXT record instead. No exposed ports needed.\nEverything stays protected behind the VPN, and Caddy handles all the certificate management automatically. Adding a new service is just a new entry in Caddy and a DNS record in Cloudflare.\nWhy Caddy? Caddy is the reverse proxy and certificate manager running on my home server. I like it above other options because it\u0026rsquo;s so straightforward and easy to configure.\nI tried nginx and Traefik, but even though nginx works great, it requires a separate certbot setup and additional configuration, which is overkill for a simple setup. Traefik has great container orchestration features, but adds configuration complexity I don\u0026rsquo;t need for what is essentially a simple reverse proxy.\nCaddy handles the entire certificate lifecycle automatically. Since the home server isn\u0026rsquo;t publicly reachable, it uses DNS-01 challenges with the Cloudflare API instead of HTTP-01. Caddy creates a temporary TXT record in Cloudflare to prove domain ownership, gets the certificate, then cleans up the record.\nSome Alternatives Here are some alternatives worth knowing:\nCloudflare Tunnels skip the VPS entirely by routing traffic through Cloudflare\u0026rsquo;s network. Setup is extremely simple (no VPS to manage) but you\u0026rsquo;re trusting Cloudflare with all your traffic.\nTraefik with Docker/Podman is worth considering if you\u0026rsquo;re running a large number of containerized services and want automatic service discovery via container labels. It\u0026rsquo;s more powerful than Caddy, just more complex to configure.\nnginx with certbot works great and gives you more control over every aspect of certificate management. Its a great choice for complex configurations and control.\n","permalink":"https://danoss.me/en/categories/security/ssl-tls-certificates-for-self-hosted-services/","summary":"Small guide and notes for SSL/TLS certificates in a self-hosted environment","title":"SSL/TLS Certificates for Self-Hosted Services"},{"content":"This part follows Linux security hardening with some system-level protections: kernel hardening parameters, SELinux access control, and audit logging that tracks everything that is happening on your system.\nSELinux Mandatory Access Control SELinux can be complicated but it\u0026rsquo;s a great kernel security feature that keeps blocking unauthorized actions on your system.\nEven if it can be challenging to configure, SELinux has great defaults for most common applications, so for 90% of use cases you won\u0026rsquo;t even notice SELinux running. If SELinux is blocking something in a default configuration, you an add a policy exception to allow the action. Do not disable SELinux.\nCheck SELinux Status First, verify SELinux is actually enabled and enforcing.\n# Check current mode getenforce # Should show \u0026#34;Enforcing\u0026#34; # Detailed status sestatus # Shows: SELinux status, policy type, mode # Check if SELinux is denying anything ausearch -m avc -ts recentEnforcing mode means SELinux actively blocks unauthorized actions. Permissive mode logs violations but doesn\u0026rsquo;t block them, which is useful for debugging but provides no actual protection.\nUnderstanding SELinux Contexts Every file, process, port, and user in SELinux has a security context (also called a \u0026ldquo;label\u0026rdquo;). These contexts determine what can access what. The format is user:role:type:level, but for most cases, you only care about the type field.\nFor example, web server files should have httpd_sys_content_t type, and the httpd process runs with httpd_t type. SELinux policy allows httpd_t processes to read httpd_sys_content_t files, but not user_home_t files (which are in user home directories). This means even if httpd is compromised and running as root, it still can\u0026rsquo;t access SSH keys in /root/.ssh/ because the SELinux policy denies it.\nView file and process contexts.\n# Show file context ls -Z /var/www/html/index.html # Output: unconfined_u:object_r:httpd_sys_content_t:s0 # Show process context ps -eZ | grep httpd # Output: system_u:system_r:httpd_t:s0 1234 ? 00:00:00 httpd # Check context of running services systemctl status httpdWhen you create or move files, they might inherit the wrong context. This is the most common cause of SELinux denials. For instance, if you copy a file from your home directory to /var/www/html/, it might keep the user_home_t context instead of getting httpd_sys_content_t.\nFixing Context Issues The restorecon command restores default contexts based on file location.\n# Restore context for a single file restorecon /var/www/html/index.html # Restore recursively for a directory restorecon -Rv /var/www/html/ # Preview what would change without applying restorecon -Rvn /var/www/html/For permanent custom contexts (like if you\u0026rsquo;re hosting web content from a non-standard location), use semanage fcontext.\n# Add persistent context rule for custom web directory semanage fcontext -a -t httpd_sys_content_t \u0026#34;/web/custom(/.*)?\u0026#34; # Apply the new rule (restores the policy) restorecon -Rv /web/custom/ # View custom context rules semanage fcontext -l | grep customSELinux Booleans SELinux uses booleans to toggle specific behaviors without rewriting policies so you can turn them on and off for specific scenarios. The -P option makes changes persistent across reboots.\n# List all booleans getsebool -a # Check specific boolean getsebool httpd_can_network_connect # Example # Allow httpd to make outbound network connections (for reverse proxies, API calls) setsebool -P httpd_can_network_connect onTroubleshooting SELinux Denials When SELinux blocks something, it logs an \u0026ldquo;AVC denial\u0026rdquo; in the audit log. These messages are cryptic, but audit2why translates them into human-readable explanations.\n# View recent denials ausearch -m avc -ts recent # Explain why access was denied ausearch -m avc -ts recent | audit2whyImportant: Don\u0026rsquo;t just blindly run audit2allow and install whatever policy it generates. Read the denial, understand what\u0026rsquo;s being blocked, and determine if the application should actually be allowed to do that. Sometimes SELinux is protecting you from a misconfigured or compromised application.\nFor example, if you see denials for httpd trying to execute files in /tmp/, the correct fix isn\u0026rsquo;t to allow it. The correct fix is to figure out why httpd is trying to execute temp files and fix the application configuration.\nAudit System Activity I\u0026rsquo;ts important to check what is happening on your system. What ports are opened, what is running, what is being accessed.\nCheck Listening Network Services Check what services are exposed on the network. Any listening port is a potential attack vector.\n# Show all listening TCP/UDP ports with process info ss -tulpn # Alternative using lsof lsof -i -P -n | grep LISTEN # Show only TCP listening ports ss -tln # Show listening ports with numeric addresses (no DNS lookups) netstat -tulnThe output shows which ports are bound to which interfaces. Pay attention to:\nPorts listening on 0.0.0.0 (all interfaces) vs 127.0.0.1 (localhost only) Unexpected services you don\u0026rsquo;t remember installing High port numbers that might be backdoors or misconfigurations For each listening port, verify:\nWhat service owns it (ss -tulpn shows the process) Whether you actually need it exposed If it should be restricted to localhost instead of all interfaces If you find services listening that you don\u0026rsquo;t recognize, investigate before disabling.\n# Identify which package owns the binary rpm -qf $(which servicename) # Check what the service does systemctl status servicename man servicenameAuditd for System Event Logging The standard system logging with journald captures service messages and errors, but it doesn\u0026rsquo;t track detailed security events like file accesses or authentication attempts. That\u0026rsquo;s what auditd does.\nAuditd works at the kernel level. It can track virtually any system activity (file accesses, system calls, network connections, user commands).\nInstall and enable auditd.\n# Should already be installed on RHEL/Fedora dnf install audit # Enable and start service systemctl enable --now auditd # Check status systemctl status auditdConfigure Audit Rules Audit rules define what events to log. You can monitor specific files, directories, system calls, or user actions. Rules are defined in /etc/audit/rules.d/audit.rules or /etc/audit/audit.rules.\nSome useful audit rules.\n# Monitor changes to system authentication files -w /etc/passwd -p wa -k passwd_changes -w /etc/shadow -p wa -k shadow_changes -w /etc/group -p wa -k group_changes -w /etc/sudoers -p wa -k sudoers_changes # Monitor SSH configuration -w /etc/ssh/sshd_config -p wa -k sshd_config_changes # Track sudo usage -a always,exit -F arch=b64 -S execve -F auid\u0026gt;=1000 -F auid!=-1 -F key=sudo_commands # Monitor file deletion by users -a always,exit -F arch=b64 -S unlink -S unlinkat -S rename -S renameat -F auid\u0026gt;=1000 -F auid!=-1 -k file_deletion # Watch critical directories -w /etc/ -p wa -k etc_changes -w /boot/ -p wa -k boot_changesThe -w flag watches a file or directory. The -p flag specifies permissions to monitor:\nr - read w - write x - execute a - attribute change Add your custom rules to /etc/audit/rules.d/custom.rules.\n# Create custom rules file vim /etc/audit/rules.d/custom.rules # Reload audit rules augenrules --load # Or restart auditd service auditd restart # Verify rules are loaded auditctl -lKernel Hardening You can configure the Linux kernel with sysctl to improve its security, networking, and performance.\nMemory Protection The /proc filesystem exposes kernel memory addresses and other sensitive information. By default, any process can read these, which helps attackers bypass address space layout randomization.\nHide Kernel Pointers\nThe kernel.kptr_restrict parameter controls access to kernel pointer addresses.\n# 0 = Show kernel pointers (default, insecure) # 1 = Hide from unprivileged users, show to root # 2 = Hide from everyone, including root kernel.kptr_restrict=2Setting this to 2 hides kernel pointers from everyone, which is most secure. However, some performance monitoring tools (like perf) need access to kernel symbols to function properly. If you run into issues with debugging tools, set it to 1 instead, which hides pointers from regular users but allows root access.\nRestrict Access to Kernel Logs\nKernel logs (accessible via dmesg) can leak sensitive information about hardware, kernel addresses, and system configuration. Restrict access to root only.\n# Prevent regular users from reading kernel ring buffer kernel.dmesg_restrict=1This should be enabled by default on modern systems, but it\u0026rsquo;s worth verifying, especially on older installations or custom kernels.\nNetwork Security Parameters Several kernel networking parameters improve security against common network-based attacks:\nSYN Flood Protection\nSYN floods are a type of denial-of-service attack that exploits the TCP three-way handshake. Enabling SYN cookies allows the kernel to handle SYN floods without maintaining state for every connection attempt.\n# Enable SYN cookies net.ipv4.tcp_syncookies=1This is usually enabled by default, but verify it\u0026rsquo;s set. There\u0026rsquo;s no real downside to SYN cookies in modern kernels.\nReverse Path Filtering (Anti-Spoofing)\nReverse path filtering verifies that incoming packets actually came from the network they claim to. This prevents IP spoofing attacks.\n# Enable strict reverse path filtering on all interfaces net.ipv4.conf.all.rp_filter=1 net.ipv4.conf.default.rp_filter=1Mode 1 is strict (it drops packets if the return path doesn\u0026rsquo;t match the incoming interface. Mode 2 is loose) it only checks if the source IP is routable. For most systems, strict mode is appropriate. If you\u0026rsquo;re doing asymmetric routing (traffic comes in one interface and goes out another), you might need loose mode.\nDisable IP Forwarding\nUnless your system is a router, IP forwarding should be disabled.\n# Disable IPv4 forwarding net.ipv4.ip_forward=0 # Disable IPv6 forwarding net.ipv6.conf.all.forwarding=0This prevents your system from routing packets between network interfaces, which could be exploited to bypass firewall rules or conduct man-in-the-middle attacks.\nDisable IPv6 (If Not Using It)\nIf you\u0026rsquo;re not using IPv6, disable it entirely to reduce attack surface.\n# Disable IPv6 on all interfaces net.ipv6.conf.all.disable_ipv6=1 net.ipv6.conf.default.disable_ipv6=1Be aware this breaks applications that try to bind to IPv6 addresses. Most applications handle this gracefully, but some (like certain database connectors) might fail. Test thoroughly before deploying this in production.\nIgnore ICMP Redirects\nICMP redirects tell hosts to use a different route for certain destinations. Accepting these from untrusted networks can be used for man-in-the-middle attacks.\n# Ignore ICMP redirect messages net.ipv4.conf.all.accept_redirects=0 net.ipv4.conf.default.accept_redirects=0 net.ipv6.conf.all.accept_redirects=0 net.ipv6.conf.default.accept_redirects=0 # Don\u0026#39;t send ICMP redirects (if not a router) net.ipv4.conf.all.send_redirects=0 net.ipv4.conf.default.send_redirects=0Apply Sysctl Settings Create a custom sysctl configuration file for all your hardening parameters.\n# Create hardening configuration sudo tee /etc/sysctl.d/99-hardening.conf \u0026lt;\u0026lt; EOF # Memory protection kernel.kptr_restrict=2 kernel.dmesg_restrict=1 # Network security net.ipv4.tcp_syncookies=1 net.ipv4.conf.all.rp_filter=1 net.ipv4.conf.default.rp_filter=1 net.ipv4.ip_forward=0 net.ipv4.conf.all.accept_redirects=0 net.ipv4.conf.default.accept_redirects=0 net.ipv4.conf.all.send_redirects=0 net.ipv4.conf.default.send_redirects=0 # IPv6 security (if using IPv6) net.ipv6.conf.all.forwarding=0 net.ipv6.conf.all.accept_redirects=0 net.ipv6.conf.default.accept_redirects=0 EOF # Load the new settings sudo sysctl -p /etc/sysctl.d/99-hardening.conf # Verify settings are applied sysctl -a | grep kptr_restrict sysctl -a | grep dmesg_restrict sysctl -a | grep tcp_syncookiesSettings in /etc/sysctl.d/ persist across reboots. The 99- prefix ensures this file loads after other sysctl configs, so your settings take precedence. This is a good approach because we add it in the drop-in directory instead of writing it directly to /etc/sysctl.\nAlways Use a VPN! Always use a paid (or self-hosted) VPN for all your connections.\nCommercial VPN Services\nWhen using a VPN you are trusting the VPN provider instead of your ISP. Providers like Mullvad or ProtonVPN are good choices as they are zero-knowledge and privacy friendly.\nSelf-Hosted VPN (Tailscale/Headscale)\nFor some connections I also recommend Headscale or Tailscale, which creates a mesh VPN between all my devices. Its purpose is not to hide traffic but to create a secure connections between my devices regardless of what network they\u0026rsquo;re on.\nWhat\u0026rsquo;s Next? These are the most common ways to secure a Linux system, however security is a never-ending process and there is always new vulnerabilities and changes happening so keep this practice up-to-date.\nPerfect security doesn\u0026rsquo;t exist, so keep at it.\n","permalink":"https://danoss.me/en/categories/linux/linux-security-hardening-part-3/","summary":"Complete guide to Linux security hardening: SELinux, system audit, kernel hardening, VPN usage, Part 3 of Linux security hardening series","title":"Linux Security Hardening Part.3 - SELinux, Audit, Kernel Hardening, VPNs"},{"content":"Part 2 of Linux security hardening, here I\u0026rsquo;ll talk about securing SSH, network-facing components, firewall configuration, and some good system maintenance options.\nSSH Server Hardening Protecting SSH is key to prevent access to a system. Default SSH configurations are designed for convenience, not security. If not protected and exposed, many scripts/bots and will attempt to login to your system.\nDisable password-based authentication entirely, restrict access to specific users, and add some basic rate limiting to reduce log noise from automated brute-force attacks.\nDisable Password Authentication Password authentication over SSH is insecure. Passwords can be brute-forced, phished, or leaked in data breaches. SSH key authentication, on the other hand, relies on keypairs which is much more secure.\nThe private key stays on your local machine and never leaves. The public key gets copied to the server. When you connect, the server challenges your client to prove it has the corresponding private key without ever transmitting the key itself. No password is sent over the network, and there\u0026rsquo;s nothing for an attacker to brute-force.\nGenerate an SSH key pair on your local machine (not the server).\n# Ed25519 is the modern standard (shorter keys, better performance)w ssh-keygen -t ed25519 -C \u0026#34;your_email@example.com\u0026#34; # If you need compatibility with older systems, use RSA with 4096-bit keys ssh-keygen -t rsa -b 4096 -C \u0026#34;your_email@example.com\u0026#34; # Keys are saved to ~/.ssh/id_ed25519 (private) and ~/.ssh/id_ed25519.pub (public)Copy your public key to the server.\n# The easy way (automatically adds to ~/.ssh/authorized_keys) ssh-copy-id username@your-server.comTest key-based authentication in a \u0026ldquo;new terminal window\u0026rdquo; before making changes.\n# This should log you in without asking for a password ssh username@your-server.com # If it still asks for a password, check server logs for debugging ssh -v username@your-server.comOnce its working, disable password authentication.\n# Edit SSH daemon configuration sudo vim /etc/ssh/sshd_config # Set these directives to disable password auth and responses PasswordAuthentication no ChallengeResponseAuthentication no # Apply changes sudo systemctl restart sshdDisable Root SSH Login Even with SSH key authentication, allowing direct root login over SSH is unnecessary. If you need root access, log in as your regular user and use sudo. This creates an audit trail showing who ran what command as root, which password-less root login completely bypasses.\nI covered disabling the root account entirely in Part 1, but even if you keep root enabled for local console access, you should block it over SSH.\n# In /etc/ssh/sshd_config PermitRootLogin no # Restart sshd to implement changes sudo systemctl restart sshdSome people use PermitRootLogin prohibit-password which allows root login with keys but not passwords. I don\u0026rsquo;t do this as i find no real reason to SSH login as root.\nRestrict SSH Access to Specific Users By default, any user account on the system can attempt to SSH in. That\u0026rsquo;s fine for single-user systems, but on multi-user servers you probably only want a few accounts to have remote access.\nThe AllowUsers and AllowGroups directives let you whitelist exactly who can connect via SSH.\n# In /etc/ssh/sshd_config # Method 1: Whitelist specific users AllowUsers cloud admin # Method 2: Whitelist a group AllowGroups sshusers # You can combine both directives if neededIf using groups, create the SSH users group and add your authorized users.\n# Create SSH users group sudo groupadd sshusers # Add your user to the group sudo usermod -aG sshusers cloud # Verify group membership groups cloudThere are also DenyUsers and DenyGroups directives for blacklisting, but I prefer whitelisting with Allow directives. Explicit allow-lists are clearer and harder to misconfigure, much easier to see who has access.\nRate Limit with fail2ban Fail2ban monitors log files for failed authentication attempts and automatically creates temporary firewall rules to block repeat offenders. This reduces server load from connection attempts and cuts down on log noise from automated SSH brute-force attacks scanning the internet.\nIt\u0026rsquo;s not strictly necessary if you\u0026rsquo;ve already disabled password auth (since there\u0026rsquo;s nothing to brute-force), but you might still want to run it because it keeps the logs cleaner and reduces wasted CPU cycles from bots trying many failed connections per day.\nInstall fail2ban.\n# Fedora/RHEL sudo dnf install fail2ban fail2ban-firewalld # Enable and start service sudo systemctl enable --now fail2banCreate a local /etc/fail2ban/jail.local config file.\n# Create local override file sudo vim /etc/fail2ban/jail.local # Add SSH jail configuration [DEFAULT] # Ban duration in seconds (10 minutes) bantime = 600 # Time window to count failures (10 minutes) findtime = 600 # Number of failures before ban maxretry = 5 # Ignore localhost and your trusted IPs ignoreip = 127.0.0.1/8 ::1 192.168.1.0/24 [sshd] enabled = true port = ssh logpath = /var/log/secure backend = systemdIf you changed your SSH port from the default 22, update the port.\n[sshd] enabled = true port = 2222 # Your custom SSH port logpath = /var/log/secure backend = systemdRestart fail2ban to apply the config.\n# Restart service sudo systemctl restart fail2banFail2ban works by parsing authentication logs and creating firewalld rich rules (or iptables rules, depending on your backend configuration). You can see active bans in your firewall.\n# View firewalld rich rules created by fail2ban sudo firewall-cmd --list-rich-rulesChange the Default SSH Port? I don\u0026rsquo;t change the default SSH port from 22 because laziness\u0026hellip; Changing the port does reduce log noise from automated scans, but someone who really wants access or some advanced bots will figure out when SSH is in another port.\nIf you\u0026rsquo;re behind a firewall or using something like Tailscale or Headscale (which I am), the SSH port isn\u0026rsquo;t even exposed to the public internet anyway. But if you\u0026rsquo;re running a directly internet-exposed server and want to reduce the constant scanning noise in your logs, here\u0026rsquo;s how to change it.\n# In /etc/ssh/sshd_config Port 2222 # Choose a high port number (1024-65535) # Update firewall rules (remove the firewalld service ssh on port 22 and add the custom port) sudo firewall-cmd --permanent --add-port=2222/tcp sudo firewall-cmd --permanent --remove-service=ssh sudo firewall-cmd --reload # Restart SSH sudo systemctl restart sshdThe main downside is you have to remember to specify the port every time you connect, which is annoying, but you can also specify it in as a default config your local ~/.ssh/config file.\nHost myserver HostName your-server.com Port 2222 User cloudThen you can just run ssh myserver and it automatically uses the configured port and username.\nFirewall Configuration Linux uses firewalld (on RHEL/Fedora) or ufw (on Debian/Ubuntu) as the frontend to netfilter, the kernel\u0026rsquo;s packet filtering framework. I\u0026rsquo;m using firewalld, which groups firewall rules into zones and services for easier management compared to iptables.\nThe default firewall configuration is usually pretty good, but there are some small changes I like to make.\nWhat are Firewalld Zones? Firewalld uses \u0026ldquo;zones\u0026rdquo; to group network interfaces and define trust levels. Each zone has a default policy for incoming traffic. The most commonly used zones are:\ndrop: Drop all incoming packets without reply (most restrictive) block: Reject all incoming packets with icmp-host-prohibited messages public: Default zone for untrusted networks, allows specific services only trusted: Allow all traffic (use carefully, typically for VPN interfaces) Check your current firewall configuration.\n# Show active zones and their interfaces sudo firewall-cmd --get-active-zones # Check default zone sudo firewall-cmd --get-default-zone # List all rules in current zone sudo firewall-cmd --list-all # View all zones and their settings sudo firewall-cmd --list-all-zonesRestrict your allowed ports and services to only accept traffic that you are expecting. Block everything else.\nManaging Firewall Services Firewalld comes with predefined service definitions for common applications like HTTP, HTTPS, SSH, and DNS. These are easier to work with than remembering port numbers.\n# List available predefined services sudo firewall-cmd --get-services # Allow a service permanently sudo firewall-cmd --permanent --add-service=http sudo firewall-cmd --permanent --add-service=https # Remove a service sudo firewall-cmd --permanent --remove-service=dhcpv6-client # Add a custom port if no predefined service exists sudo firewall-cmd --permanent --add-port=8080/tcp # Remove a port sudo firewall-cmd --permanent --remove-port=8080/tcp # Reload firewall to apply all changes sudo firewall-cmd --reloadThe --permanent option writes changes to disk so they persist across reboots. Without it, rules only apply until the firewall is reloaded or the system reboots.\nICMP Filtering ICMP is used for network diagnostics like ping and path MTU discovery. The common security advice is to \u0026ldquo;disable ping\u0026rdquo; by blocking ICMP echo-request packets. This prevents someone from easily enumerating active hosts on your network through ping sweeps.\nBeware as blocking ICMP can break legitimate network functions, but allowing all ICMP types enables network reconnaissance. So it\u0026rsquo;s a trade-off like everything in life.\nIf you want to block ping requests.\n# Block ICMP echo-request (incoming pings) sudo firewall-cmd --permanent --add-icmp-block=echo-request # Block echo-reply as well (responses to outgoing pings) sudo firewall-cmd --permanent --add-icmp-block=echo-reply # Apply changes sudo firewall-cmd --reload # Test from another machine ping your-server.com # Should timeoutPersonally, I don\u0026rsquo;t block ICMP as troubleshooting network issues without ping is a pain and port scanners don\u0026rsquo;t rely only on ping.\nEnable Firewall Logging By default, firewalld doesn\u0026rsquo;t log dropped packets. Enabling logging lets you see what\u0026rsquo;s being blocked, which is useful for troubleshooting misconfigurations and detecting scanning activity from bots:\n# Log all dropped packets sudo firewall-cmd --set-log-denied=all sudo firewall-cmd --permanent --set-log-denied=all # Options: off, all, unicast, broadcast, multicast # \u0026#39;unicast\u0026#39; is less noisy than \u0026#39;all\u0026#39; but still useful # View dropped packets in system logs sudo journalctl -f -u firewalld sudo tail -f /var/log/messages | grep -i REJECTAutomated Security Updates Apply security updates as soon as they become available with either a systemd timer or dnf-automatic.\nSome might not like it but I\u0026rsquo;m lazy.\nDNF Automatic DNF Automatic is the official tool in Fedora and RHEL-based systems to automate the download and install of packages.\nI use it to automatically install security updates while leaving regular feature updates for manual review.\nInstall DNF Automatic.\n# Install package sudo dnf install dnf-automaticConfigure update behavior in /etc/dnf/automatic.conf.\n# Edit configuration sudo vim /etc/dnf/automatic.conf # Key settings to modify: [commands] # What to do with updates # apply = download and install # download = download only # check = just check for updates upgrade_type = security apply_updates = yes [emitters] # How to get notified emit_via = stdio [email] # Email configuration if you want email notifications (requires postfix or similar) email_from = root@localhost email_to = cloud@example.com email_host = localhostI set upgrade_type = security and apply_updates = yes. This automatically installs security updates but leaves regular feature updates for manual review. You might prefer upgrade_type = default to install everything automatically, which is very convenient.\nEnable and start the systemd timer.\n# Enable timer (runs daily by default) sudo systemctl enable --now dnf-automatic.timer # Check timer status systemctl status dnf-automatic.timer # View timer schedule systemctl list-timers dnf-automatic.timer # Manually trigger an update check sudo systemctl start dnf-automatic.serviceCheck logs to verify it\u0026rsquo;s working.\n# View DNF automatic logs sudo journalctl -u dnf-automatic.service # Last run output sudo journalctl -u dnf-automatic.service -n 50Flatpak Automatic Updates If you are using Flatpaks and want automated updates you can use Gnome\u0026rsquo;s software center app to turn on auto-updates or just create a simple systemd timer to update at regular intervals.\n# Create systemd service for Flatpak updates sudo vim /etc/systemd/system/flatpak-update.service [Unit] Description=Update Flatpak packages After=network-online.target Wants=network-online.target [Service] Type=oneshot ExecStart=/usr/bin/flatpak update -yCreate a timer to run it.\n# Create systemd timer sudo vim /etc/systemd/system/flatpak-update.timer [Unit] Description=Daily Flatpak update [Timer] OnCalendar=daily Persistent=true [Install] WantedBy=timers.targetEnable the timer.\n# Enable timer sudo systemctl enable --now flatpak-update.timerRemove Unused Services Every running service is a potential attack surface. If you\u0026rsquo;re not actively using a service, disable it.\nCheck which services are currently running and enabled:\n# List all enabled services (will start on boot) systemctl list-unit-files --state=enabled # List currently running services systemctl list-units --type=service --state=runningBefore disabling a service, check what depends on it to make sure you\u0026rsquo;re not breaking something critical:\n# See what would be affected by disabling a service systemctl list-dependencies --reverse servicename.service # Check if anything is actively using it systemctl status servicename.serviceGoogle (or as AI) about unfamiliar services before disabling them. Some services have very weird names but are actually legit. A quick web search for \u0026ldquo;what is [servicename] Fedora\u0026rdquo; usually show a lot of info to see if it\u0026rsquo;s safe to disable.\nDisable services you don\u0026rsquo;t need.\n# Disable and stop a service sudo systemctl disable --now ModemManager.serviceWhat\u0026rsquo;s Next? Link to Part 1: Linux Security Hardening Part.1 - LUKS, PAM, and User Lockdown.\nNext part should be SELinux, kernel hardening, systemd security\u0026hellip;\n","permalink":"https://danoss.me/en/categories/linux/linux-security-hardening-part-2/","summary":"Complete guide to Linux security hardening: SSH hardening, firewall configuration, automatic security updates, removal of unused services, Part 2 of Linux security hardening series","title":"Linux Security Hardening Part.2 - SSH, Firewall, and Services"},{"content":"Having basic security hygiene is essential, even though it\u0026rsquo;s a never ending process and perfect security doesn\u0026rsquo;t exist.\nThis topic is very extensive and is constantly evolving. So decided to split it into multiple parts. This part covers only full disk encryption, user account lockdown, and PAM configuration for user authentication.\nFull Disk Encryption with LUKS Data-at-rest encryption is a must. If someone gets physical access to your hardware, full disk encryption is the only thing protecting your data from being read directly off the drive.\nLinux uses LUKS (Linux Unified Key Setup) for disk encryption.\nWhat is LUKS? LUKS adds an encryption layer between the physical block device (the hard drive\u0026rsquo;s representation inside Linux) and the filesystem (ext4, xfs, btrfs\u0026hellip;).\nIt uses the kernel\u0026rsquo;s device mapper to create a mapping between the encrypted physical drive and a decrypted logical volume. When you\u0026rsquo;re using your computer, applications see just a regular decrypted device in /dev/mapper/. They\u0026rsquo;re unaware of the encryption.\nWhen you encrypt a drive, LUKS adds a header with metadata about the encryption (cipher, key size, key derivation function) and up to 32 key slots in LUKS2. These key slots can store passphrases or key files that unlock the same encrypted volume, so they can be used by multiple users to decrypt the drive using their own credentials stored in separate slots.\nKeep in mind that to access a LUKS-encrypted volume, you need to decrypt it first. So data is only encrypted when the volume is closed (manually locked or when you shut down the computer). While you\u0026rsquo;re working with the volume, data is decrypted in memory.\nLUKS1 vs LUKS2\nIn modern systems LUKS2 is the default, which provides more keyslots and some improvements.\nUp to 32 keyslots (vs 8 in LUKS1) Argon2 key derivation (resistant to GPU-based attacks) Authenticated encryption support Better header resilience Online re-encryption support (encrypt existing data without reformatting) Key Derivation Functions\nThe key derivation function determines how your passphrase is converted into an encryption key:\nPBKDF2 (LUKS1 default): Fast but vulnerable to GPU-accelerated brute-force attacks Argon2i (LUKS2 default): Memory-hard algorithm that resists GPU attacks and cache-timing attacks Argon2id: Hybrid approach with best overall protection against both GPU and side-channel attacks Setting Up LUKS Encryption You cannot encrypt a drive that\u0026rsquo;s currently in use. You need either a new drive or if using an existing one, backup the data, format the drive, encrypt it, and then restore.\nMost Linux distros offer LUKS setup during install (usually just by clicking a checkbox). You\u0026rsquo;ll just be asked to wipe the drive and enter a passphrase. This is the easiest approach.\nsource: image found in fedoramagazine\nEncrypt a Drive After Install\n# Check if a device is already LUKS-encrypted cryptsetup isLuks -v /dev/sdb1 # Format the device with LUKS encryption (you\u0026#39;ll be prompted for a passphrase) cryptsetup luksFormat /dev/sdb1 # Use specific cipher and key size for more control cryptsetup luksFormat --cipher aes-xts-plain64 --key-size 512 /dev/sdb1 # Specify key derivation function cryptsetup luksFormat --type luks2 --pbkdf argon2id --pbkdf-memory 1048576 /dev/sdb1Create LUKS Volume with Detached Header\nStoring the LUKS header separately is more secure. Without access to the external header file, the encrypted drive remains inaccessible. This is kinda extreme, but you can store the header on a USB drive.\n# Create LUKS with detached header cryptsetup luksFormat /dev/sdb1 --header /root/luks-header.img # Open with detached header cryptsetup luksOpen /dev/sdb1 encrypted-data --header /root/luks-header.imgWorking with LUKS View Current LUKS Version\n# Check LUKS version cryptsetup luksDump /dev/sdb1 | grep VersionBackup LUKS Header\nIf the LUKS header gets corrupted (bad sector, filesystem corruption) your encrypted data will be unrecoverable. Back up your LUKS header immediately after encrypting any drive.\n# Backup LUKS header (do this immediately after encryption) cryptsetup luksHeaderBackup /dev/sdb1 --header-backup-file /root/luks-header-backup.img # Store backup on separate media (USB drive, password manager, cloud storage) # DO NOT store it on the encrypted drive itself # Restore corrupted header cryptsetup luksHeaderRestore /dev/sdb1 --header-backup-file /root/luks-header-backup.img # Check header integrity cryptsetup luksDump /dev/sdb1Manually Open LUKS Volume\nOnce drive is encrypted, you need to open the LUKS volume to access it contents.\n# Open the LUKS volume with a device mapper name (creates /dev/mapper/encrypted-data) cryptsetup luksOpen /dev/sdb1 encrypted-data # Format the decrypted volume with a filesystem (here, xfs) mkfs.xfs /dev/mapper/encrypted-data # Mount the volume like any other device mount /dev/mapper/encrypted-data /mnt/secureManually Close LUKS Volume\nClose the open LUKS volume to restrict access. This removes the device mapper entry to ensure the contents are no longer accessible.\n# Unmount the filesystem first umount /mnt/secure # Close the LUKS volume cryptsetup luksClose encrypted-data # Alternative shorter syntax cryptsetup close encrypted-dataWorking with Key Slots\nYou can add keys to different key slots to have multiple decryption methods (this is useful for multi-user systems or recovery).\nI just use slot 0 for my primary passphrase and slot 1 for recovery (saved in password manager).\n# View current key slots and their status cryptsetup luksDump /dev/sdb1 | grep \u0026#34;Key Slot\u0026#34; # Add a new passphrase to an available slot cryptsetup luksAddKey /dev/sdb1 # Add a keyfile instead of a passphrase cryptsetup luksAddKey /dev/sdb1 /path/to/keyfile # Add to a specific slot number cryptsetup luksAddKey --key-slot 2 /dev/sdb1 # Change an existing passphrase cryptsetup luksChangeKey /dev/sdb1 # Remove a key slot cryptsetup luksKillSlot /dev/sdb1 1Auto-Mounting with Keyfiles\nAdd key files to auto-mount the drive at boot (this is setup automatically if you add LUKS encryption at install).\nCreate keyfile.\n# Generate a random 4096-byte keyfile dd if=/dev/urandom of=/root/luks-keyfile bs=4096 count=1 # Restrict permissions (critical—prevent unauthorized access) chmod 400 /root/luks-keyfile # Add it to the LUKS volume cryptsetup luksAddKey /dev/sdb1 /root/luks-keyfileConfigure automatic decryption in /etc/crypttab.\n# Syntax: name device keyfile options encrypted-data /dev/sdb1 /root/luks-keyfile luksAdd the auto-mount entry in /etc/fstab.\n# Use the device mapper name in /etc/fstab /dev/mapper/encrypted-data /mnt/secure xfs defaults 0 0The LUKS volume now automatically decrypts and mounts at boot. Data remains encrypted when the system is powered off or when during pre-boot stages.\nEncrypted Swap\nIf you\u0026rsquo;re using swap space, you must also encrypt it. Else, sensitive data that gets paged to swap during your session stays there unencrypted on disk.\n# Random key for swap (regenerated each boot) # Add to /etc/crypttab swap /dev/sda2 /dev/urandom swap,cipher=aes-xts-plain64,size=512 # In /etc/fstab /dev/mapper/swap none swap defaults 0 0User Account Security Disable Root Login Running as root is not a good idea. Disable the root account and just use a regular user with sudo privileges instead.\nThis prevents anyone from logging in as root, either at the console or via SSH. You can still get root privileges through sudo or by using sudo -i for an interactive root shell.\n# Lock the root account passwd -l root # Deny root login via SSH /etc/ssh/sshd_config vim /etc/ssh/sshd_config PermitRootLogin noSession Timeouts Auto-logout inactive users in /etc/profile.\n# Add to /etc/profile TMOUT=900 # Auto-logout after 15 minutes of inactivity readonly TMOUT export TMOUTPassword History Prevent users from cycling through and reusing recent passwords. Most distros remember the last 10 passwords by default, so you can increase this number to prevent users from reusing old passwords. A little extreme for me but good to know.\nChange the config in /etc/security/pwhistory.conf.\n# Increase to 24 (common in enterprise Linux environments) remember = 24 enforce_for_rootRestrict Home Directory Permissions The default home directory permissions on some distros are too permissive, as \u0026ldquo;other\u0026rdquo; users can often read files in your home directory.\nChange existing home directory permissions.\n# Remove read access for others chmod o-r $HOME # More explicitly, set to owner-only access chmod 700 $HOMESet restrictive defaults for new users in /etc/login.defs.\n# Set restrictive umask for new files UMASK 077 # Set home directory permissions to owner-only HOME_MODE 0700Set restrictive umask for existing users\u0026rsquo; new files in ~/.bashrc (or better, in the drop-in directory ~/.bashrc.d/).\n# Set umask for new files created by this user umask 027 # Or more restrictive (owner-only access) umask 077Configure Sudo Access sudo allows configured users to execute commands with elevated privileges or as another user without knowing the target user\u0026rsquo;s password. Ensure this is properly configured.\nEdit /etc/sudoers with visudo, which validates syntax before saving.\n# Edit /etc/sudoers visudo # Better: create a custom config file in the sudoers drop-in directory visudo -f /etc/sudoers.d/customSome good sudo security settings:\n# Require password for every sudo command Defaults timestamp_timeout=0 # Or set a 5-minute timeout fo asking the user for the password again if too annoying Defaults timestamp_timeout=5 # Use sudo aliases # Syntax: User_Alias NAME = user1, user2, %group1, ... User_Alias ADMINS = alice, bob, %sysadmin # Syntax: Host_Alias NAME = host1, host2, ... Host_Alias PRODUCTION = prod-web01, prod-db01, 10.0.1.0/24 # Syntax: Cmnd_Alias NAME = /path/to/command1, /path/to/command2, ... # Very important! When commands are listed with specific arguments, only those exact argument combinations are permitted. Also, don\u0026#39;t use wildcards as for example, a command like `/usr/bin/systemctl * sshd` allows `systemctl stop sshd`, `systemctl restart sshd`, etc. Dangerous! Cmnd_Alias SERVICES = /usr/bin/systemctl, /usr/sbin/service Cmnd_Alias SOFTWARE = /usr/bin/dnf, /usr/bin/yum, /usr/bin/rpm # All together, sudo rule using aliases ADMINS PRODUCTION=(ALL) SOFTWARE, SERVICESCheck you user\u0026rsquo;s sudo privileges.\n# List your own sudo privileges sudo -lPassword Aging Policy Set password aging for new users in /etc/login.defs.\n# Max password age in days PASS_MAX_DAYS 90 # Min days between password changes (prevents immediate cycling back to old passwords) PASS_MIN_DAYS 1 # Days before expiration to warn user PASS_WARN_AGE 7 # Min password length (legacy setting, use pwquality instead) PASS_MIN_LEN 8Configure password aging for existing users:\n# View current password aging settings chage -l username # Set password to expire in 90 days chage -M 90 username # Set minimum days between password changes chage -m 7 username # Force password change at next login chage -d 0 username # Set account expiration date chage -E 2025-12-31 username # Interactive configuration mode chage usernameUnderstanding PAM Understanding (at least at a high-level) PAM is necessary.\nPAM is a centralized authentication framework that many applications use to verify user credentials. Without PAM, every application would implement its own authentication logic.\nThe PAM authentication process works like this:\nApplication requests authentication through PAM API PAM reads configuration from /etc/pam.d/ for that specific service PAM loads and executes configured modules in order Each module performs its specific authentication check PAM combines results and tells the application success or failure PAM modules fall into four types:\nauth - Verify credentials (passwords, tokens, biometrics, YubiKeys) account - Check account status (expiration, access times, lockouts) password - Handle password changes and enforce complexity rules session - Set up and tear down user sessions (mount home directories, set limits) PAM control flags determine what happens when a module succeeds or fails:\nrequisite - Must succeed or authentication fails immediately, no further modules checked sufficient - If this succeeds, skip remaining modules of this type required - Must succeed, but PAM continues checking other modules regardless optional - Module result only matters if it\u0026rsquo;s the only module in the stack include - Include all lines of given type from another configuration file Enforce Password Complexity Password Complexity with pwquality\nThe pam_pwquality PAM module enforces password complexity requirements through PAM. It uses a \u0026ldquo;credit\u0026rdquo; system for defining password strength (you\u0026rsquo;ll have to specify the number of required digits, uppercase letters, lowercase letters, and special characters).\nHow it works is kind of confusing but, for example, for minlen = 12 and you set dcredit = 1, a password with one digit gets 1 credit toward the minimum length, so it needs 11 regular characters plus the digit. But if ucredit = -1, uppercase is required but doesn\u0026rsquo;t reduce the length requirement, you still need 12 characters total including the uppercase.\nConfigure it in /etc/security/pwquality.conf.\n# Credit system clarification: # Positive value (dcredit = 1): \u0026#34;If password has 1+ digits, reduce minlen by 1\u0026#34; # Negative value (ucredit = -1): \u0026#34;Password MUST have 1+ uppercase, no length credit\u0026#34; # Example with minlen = 12: # dcredit = 1 → Password with digit can be 11 chars total # ucredit = -1 → Password must have uppercase AND still be 12 chars dcredit = -1 # Require at least 1 digit ucredit = -1 # Require at least 1 uppercase lcredit = -1 # Require at least 1 lowercase ocredit = -1 # Require at least 1 special character # Min password length (considering credits) minlen = 12 # Max consecutive identical characters maxrepeat = 2 # Max consecutive characters from same character class maxclassrepeat = 3 # Number of password entry attempts before giving up retry = 3 # Min different characters from old password difok = 5 # Check against dictionary words dictcheck = 1 # Reject passwords containing username usercheck = 1 # Enforce rules for root user (0=yes, 1=no) enforce_for_root = 0Integrate pwquality with PAM\nEnsure PAM uses the pwquality module by checking /etc/pam.d/passwd and /etc/pam.d/system-auth.\n# In /etc/pam.d/passwd password requisite pam_pwquality.so retry=3 password sufficient pam_unix.so sha512 shadow try_first_pass use_authtok password required pam_deny.so # In /etc/pam.d/system-auth password requisite pam_pwquality.so try_first_pass local_users_only retry=3 password sufficient pam_unix.so sha512 shadow try_first_pass use_authtok password required pam_deny.soPrevent Brute-Force Attacks The pam_faillock PAM module protects against brute-force password attacks by locking accounts after consecutive failed authentication attempts. This is important for any system, even if not exposed to the network. Note this only protects for PAM-based logins (for example, it doesn\u0026rsquo;t apply for recovery mode).\nConfigure faillock behavior in /etc/security/faillock.conf.\n# Directory for failed attempt records dir = /var/run/faillock # Number of failed attempts before lockout deny = 4 # Lockout duration in seconds (1200 = 20 minutes) unlock_time = 1200 # Apply lockout to root account even_deny_root # Root-specific lockout duration (if even_deny_root is set) root_unlock_time = 600 # Enable audit logging audit # Lock accounts if faillock can\u0026#39;t write to log directory silentTest the configuration.\n# Attempt to log in with wrong password multiple times ssh username@localhost # Check if account is locked faillock --user username # Verify you can still unlock sudo faillock --user username --resetEnable Faillock with Authselect\nOn RHEL and Fedora, use authselect profiles rather than manually editing PAM files. This prevents configuration conflicts and ensures consistency across PAM service files.\n# Show current authselect profile authselect current # List available profiles authselect list # List features of a profile authselect list-features sssd # Enable faillock feature authselect enable-feature with-faillock # Apply changes authselect apply-changesAuthselect modifies /etc/pam.d/system-auth and /etc/pam.d/password-auth to include faillock checks. The configuration looks like.\n# Auth section - check for lockout before authentication auth required pam_faillock.so preauth silent auth sufficient pam_unix.so nullok try_first_pass auth [default=die] pam_faillock.so authfail auth required pam_deny.so # Account section - enforce lockout account required pam_faillock.so account required pam_unix.soView Failed Login Attempts\n# Show all failed attempts system-wide faillock # Show failed attempts for specific user faillock --user username # More detailed information faillock --user username --verboseLogin with YubiKey I\u0026rsquo;ve added PAM rules to login via YubiKey for more security.\nMore details here: Using YubiKey for Local Linux Authentication\nWhat\u0026rsquo;s Next? In the end, security hardening and privacy is not perfect. There are extreme measures that make you very secure, but makes using the system painful. This is a good balance on usability and security.\nNext parts will cover SSH, networking, kernel, LSM (SELinux specifically), systemd, firewall configuration \u0026hellip;\n","permalink":"https://danoss.me/en/categories/linux/linux-security-hardening-part-1/","summary":"Complete guide to Linux security hardening: LUKS full disk encryption, PAM configuration, user account lockdown, and brute-force protection, Part 1 of Linux security hardening series","title":"Linux Security Hardening Part.1 - LUKS, PAM, and User Lockdown"},{"content":"Finally Firefox supports XDG directories!\nAfter years of cluttering home directories with .mozilla folders, Firefox 147 finally adopted the XDG Base Directory specification. Even though the change is small it is a great one for organization freaks.\nApps that don\u0026rsquo;t follow the XDG directory specification create disorder in user\u0026rsquo;s home directories by creating a dotfile directory for each. For example, .mozilla could be a directory next to .cache, next to .nmp, etc.\nWhat\u0026rsquo;s XDG? The XDG Base Directory spec comes from freedesktop.org to create a consistent structure for where applications should store their files on Linux systems. Instead of each program creating its own hidden directory directly in your home folder, XDG defines specific locations based on file type.\nConfig files belong in ~/.config. Application data (things like databases, profiles, state) goes in ~/.local/share. Cache files go to ~/.cache.\nThis makes it easy for example for backups or to get data from applications, just get everything from ~/.config and ~/.local/share. Want to clear cache? delete ~/.cache without worrying. Pretty simple.\nWhat Changed in Firefox 147 With Firefox 147, new installs use XDG-compliant paths. Your profile goes to ~/.config/mozilla/firefox, cache goes to ~/.cache/mozilla/firefox. Clean, organized, exactly where it should be.\nThis only affects fresh installs. Existing install are kept in the ~/.mozilla directory. This change doesn\u0026rsquo;t break anything. Firefox simply checks for the XDG environment variables and uses those paths for new profiles while respecting existing installations.\nShould You Migrate? There is no need to migrate. Nothing was broken to begin with and Firefox continues to work with the old path.\nI am wary that maybe some extensions might break with the new path, so if you migrate always backup the config and copy the profile data to the new location.\n","permalink":"https://danoss.me/en/categories/linux/firefox-supports-xdg-directories-on-linux/","summary":"Finally Firefox supports XDG directories!","title":"Firefox Finally Supports XDG Directories on Linux"},{"content":"Something old, but still relevant to know to avoid a simple privilege escalation path is shell escapes.\nWhat are Shell Escapes? Shell escapes are a feature of many text-based programs that let you temporarily exit the program\u0026rsquo;s interface to execute shell commands without closing the application.\nThe problem is that shell escapes don\u0026rsquo;t just give you access to a shell, it gives you access to a shell with the permissions and environment of the parent program, so if you\u0026rsquo;re running vim with sudo, any shell escape you trigger runs with root privileges, creating a potential privilege escalation vector.\nThis is not a bug, just a feature that has been around for ages but that can be dangerous if not managed.\nHow Shell Escapes Work Here\u0026rsquo;s how shell escapes work:\nA user enters the vim text editor. On the vim interface, the user triggers a shell escape (e.g., !/bin/sh) which instructs vim (the parent process) to invoke the fork() system call to create a new child process (the new shell). This child process is an exact copy of the parent process (vim), including its current state, execution environment and privileges: Environment variables (PATH, HOME, USER, etc.) Working directory (current location in filesystem) Process permissions (UID/GID, supplementary groups) File descriptors (open files, network connections) Resource limits (memory, CPU, file handles) The child process then invokes the execve() system call to replace itself with a shell (e.g., /bin/sh). The shell then starts execution with the inherited environment and permissions. So, if the parent process (e..g, vim) was executed with sudo privileges, the shell inherits these elevated privileges as well. When the user exits the shell (via exit or Ctrl+D), the shell process terminates and control returns to the parent process (vim. The parent process continues execution as if nothing happened. Programs with Shell Escapes Common Examples in Linux In a regular Linux install, shell escapes can be done in:\nText editors like vim and emacs support shell escapes. In vim, you can use :shell to open a full interactive shell, :!command to execute single commands, or even :read !command and :write !command to pipe content through shell commands. Each of these can be abused if vim is running with elevated privileges. Pager programs like less and more have their own shell escape mechanisms. In less, the !command syntax executes shell commands, the v key opens your $EDITOR (which might itself support shell escapes), and the | command syntax pipes content through shell commands. Database clients like mysql, psql, and sqlite3 often have shell escape features. Many allow you to execute system commands from within their interactive interfaces, which becomes a problem when these clients are run with elevated privileges or have access to sensitive data. The GTFOBins Repository The GTFOBins is a cool repository of Linux binaries that support shell escapes and similar privilege escalation techniques.\nIt\u0026rsquo;s a good to know resource, for example it shows that you can use less shell escapes like this. You\u0026rsquo;re now root, because less executed the shell with its privileges.\nsudo less /etc/hosts # Inside less, press ! followed by the command (syntax: !command) !/bin/sh sh-5.3 # New promptContainer Escapes Something i think of like an evolution of shell escapes are container escapes. They both involve breaking out of isolation to reach a more privileged environment. Instead of a shell breakout like vim, container escapes break out of container isolation to access the host.\nContainers are common nowadays and a lot of them run AI/ML workloads, which makes them a good target for exploit, as some need privileged access to hardware resources like GPU, require socket mounts, or some dangerous kernel capabilities to work as intended.\nDefending Against Shell Escapes Going back to shell escapes\u0026hellip; Defending against shell escapes is straight forward but important, there are many approaches to be protected.\nPrinciple of Least Privilege Always grant the minimum access necessary to users to accomplish tasks.\nFor sudo access, grant access to specific commands with specific arguments, not broad access to interactive programs. For example, in the sudoers file, grant access only to sudoedit for file editing tasks instead of granting overall sudo access to text editors. Use the NOEXEC tag to prevent command execution when it\u0026rsquo;s not necessary.\nFor containers, run with the minimum necessary capabilities. Drop all capabilities and add back only what\u0026rsquo;s required. Never run privileged containers, use rootless containers instead.\nsudoedit For file editing, grant sudoedit access to users instead of plain sudo:\n# In /etc/sudoers - wrong: user ALL=(ALL) /usr/bin/vim /etc/ssh/sshd_config # Correct way: user ALL=(ALL) sudoedit /etc/ssh/sshd_configsudoedit creates a temporary copy of the target file owned by the user. It opens that copy in the user\u0026rsquo;s text editor running with the user\u0026rsquo;s normal privileges. When you save and exit, sudoedit copies the modified file back to the original location with root privileges.\nSince the editor never runs as root, shell escapes only give you access to your own user account.\nNOEXEC In sudoers, use the NOEXEC tag for commands that need to run with elevated privileges but shouldn\u0026rsquo;t be able to execute other programs:\n# Contents of /etc/sudoers user ALL=(ALL) NOEXEC: /usr/bin/vim /etc/config user ALL=(ALL) NOEXEC: /usr/bin/less /var/log/syslogNOEXEC sets the LD_PRELOAD environment variable to load a library that intercepts calls to execve(). When a shell escape tries to spawn a new process, the call is blocked.\nSome programs might use system calls that aren\u0026rsquo;t intercepted, and it can break programs that legitimately need to execute other commands. But for many use cases, it provides a good protection against shell escapes.\nConclusion Shell escapes are not new, but they\u0026rsquo;re still relevant as they represent how privilege escalation can happen through everyday features. Sometimes a convenient functionality can lead to dangerous consequences.\nThe principle of least privilege is always a good welcome. Prevent sudo exploitation by granting access to specific commands, use sudoedit for file editing, apply NOEXEC where appropriate, and think about what users and containers actually need. Most privilege escalation attacks succeed because someone granted more access than necessary.\nRelated topics:\nsudo su - Why Embrace Redundancy? Kitty Terminal Emulator Kernel Panic!!! ","permalink":"https://danoss.me/en/categories/linux/shell-escapes/","summary":"Some notes on shell escapes on Linux","title":"Shell Escapes"},{"content":"Today I was cleaning up some old files on my home computer and found my local backup setup script. I set it up more than a year ago using restic and Backblaze B2 for cloud storage.\nRestic is a fast, secure backup program for Linux that uses encryption and deduplication to create efficient snapshots of your files to local or remote storage. Backblaze B2 is an S3-compatible cloud storage service designed for backups and archives, offers cheaper storage than AWS or Google Cloud. I\u0026rsquo;m a believer in the setup once and forget mentality. The script backs up selected files and directories from my computer to B2 cloud storage every 2 weeks and sends me a notification once the backup operation is done and the restic checks pass. Every 6 months or so I\u0026rsquo;ll poke at it to make sure backups are actually there and working.\nLuckily I\u0026rsquo;ve never had to do a full restore, but the setup works and I\u0026rsquo;m not worried about losing my stuff if my hard drive dies tomorrow. This is just for my main computer, nothing server-related. Just personal files that I\u0026rsquo;d be annoyed to lose.\nWhy Restic and B2? Yes, I could use Microsoft\u0026rsquo;s OneDrive, Google Drive, Dropbox, etc. but I don\u0026rsquo;t like their privacy practices and the fact they are constantly running. I just need a simple automated backup solution with minimal footprint on my machine.\nRestic encrypts everything with AES-256, deduplicates data automatically, and works with basically any storage backend.\nBackblaze B2 is I believe the cheapest cloud storage option (compared to Cloudflare R2, AWS S3, or Google Cloud) and is pretty reliable. I trust the Backblaze team. For personal backups where I\u0026rsquo;m hopefully never downloading anything, this works out great. First 3x your storage is free for downloads, then it\u0026rsquo;s $0.01/GB after that.\nPlus B2 is S3-compatible, which means if I ever want to switch to Cloudflare R2 or whatever, I just change the repository URL. Same script, different backend (I\u0026rsquo;m not locked into any provider).\nThe Setup The setup is just a shell script that checks that config files exist, loads the credentials, and verifies the backup is done correctly by running restic check at the end.\nScript: restic-backup.sh\nWhat Gets Backed Up?\nConfiguration files specify which directories should be backed up and which patterns should be excluded. If I need to modify the existing backup paths just edit the file.\n# Example backup path config file ~/Documents ~/Pictures ~/.sshSecrets Management\nSecrets (B2 API keys and restic repository password) are stored in a file with 0600 permissions, so only my local user has access to it. This is not the most secure way to handle secrets, but I\u0026rsquo;ve been lazy to migrate to a more secure option.\n# Example secrets B2_ACCOUNT_ID=\u0026#34;0123456789abcdef01234567\u0026#34; B2_ACCOUNT_KEY=\u0026#34;K001aBcDeFgHiJkLmNoPqRsTuVwXyZ\u0026#34; RESTIC_REPOSITORY=\u0026#34;b2:my-backup-bucket:restic\u0026#34;Automation\nSystemd makes things easy. I just created a .service unit for executing the script and a .timer unit to run the service at regular intervals.\nThe service runs as my regular user account so no root access is needed, but you can run as root if needed.\nOnCalendar=*-*-01,15 02:30:00 Persistent=true RandomizedDelaySec=30minOnce the service is executed, I have a custom binary that notifies me over Signal the backup details and execution status. This is optional, but convenient step. You can also see the output in the systemd journal.\nOutput\nThe resulting output looks something like this\n============================================= BACKUP SUCCESSFUL ============================================= Timestamp: 2026-01-12 01:26:01 Repository: b2:MyBucket:repo_dir Source Dirs: /home/user/.bashrc.d/ /home/user/Backup/ -------------- Restic Summary --------------- Files: 0 new, 1 changed, 12831 unmodified Dirs: 0 new, 4 changed, 4701 unmodified Data Added: 180KiB Total Processed: 232MiB Duration: 0m 6s Snapshot ID: e6011222 --------------------------------------------- Verifying backup integrity... ✓ Backup verification successfulWhy This? This is not the best backup management system but I don\u0026rsquo;t need web interfaces, GUI, synchronization across devices, or many other features. Backing up a single main computer should be simple, with minimal dependencies, and invisible. This has it all, even if there are more secure and fully featured options out there.\nWould I migrate to another backup solution? In the future, yes. Secret security needs to improve and some QoL features would be nice, hopefully I can make time to improve the script. Hope I don\u0026rsquo;t forget.\n","permalink":"https://danoss.me/en/categories/linux/automated-backups-restic-and-backblaze-b2/","summary":"Just a simple script to automate local system backups with restic and Backblaze B2 cloud storage on a Linux system","title":"Automated Backups with Restic and Backblaze B2"},{"content":"Having a VPN in today\u0026rsquo;s world is a must, and I can\u0026rsquo;t recommend Mullvad VPN enough. I\u0026rsquo;ve been using it for years now and am very satisfied (not paid sponsor).\nMullvad VPN is serious about privacy, zero-knowledge, requires no email registration, and you can pay with crypto (and even better, Monero). Under the hood, Mullvad runs on WireGuard, which means low overhead, fast handshakes, and a minimal attack surface compared to OpenVPN.\nMullvad Clients Linux Client: Mullvad has a Linux client that works great on Fedora (and any other distro). It\u0026rsquo;s simple to use and works great. Good GUI for normal configurations and a CLI client for advanced settings.\nAndroid Client: Mullvad also has a really good Android client app (don\u0026rsquo;t like Apple ecosystem jail), which works great, simple, and fully featured.\nRunning Multiple VPNs Normally, you would use only one VPN for all traffic, however I need to use 2 VPNs at once:\nMullvad to protect all my network traffic Tailscale to access my private services available on a public VPS. The VPS runs a Headscale server that allows connections only from authenticated Tailscale clients. The problem is fundamental: both VPNs compete for the default route. When Mullvad connects, it installs a default route through its WireGuard interface (wg0) and uses policy-based routing to force all traffic through it. Tailscale installs its own routes for the 100.64.0.0/10 CGNAT range and expects direct connectivity to its coordination server. When Mullvad\u0026rsquo;s catch-all routing rule wins, Tailscale\u0026rsquo;s control plane traffic gets tunneled through Mullvad, which breaks authentication and peer discovery. Mullvad does offer split tunneling to selectively exclude app traffic from the tunnel, but it works by cgroup or PID matching, not by destination address. Since Tailscale is a kernel-level WireGuard peer rather than a regular userspace application, app-based exclusions don\u0026rsquo;t apply cleanly. You can\u0026rsquo;t tell Mullvad \u0026ldquo;exclude the Tailscale interface\u0026rdquo; through the GUI.\nUsing Both Mullvad and Tailscale I researched online for a solution and found that Tailscale offers a paid add-on for allowing Tailscale traffic over Mullvad via exit nodes, so, I decided to create a simple nftables rule to allow both connections on Linux.\nHow Mullvad Split Tunneling Works Mullvad\u0026rsquo;s split tunneling is implemented entirely in nftables using packet marks, not application-level interception. Two marks control the behavior:\n0x00000f41: applied to the connection tracking entry (ct mark), causing Mullvad\u0026rsquo;s conntrack rules to not capture the flow 0x6d6f6c65: applied to the packet itself (meta mark), directing the kernel\u0026rsquo;s policy routing to skip the Mullvad routing table When Mullvad sees these marks on a packet, its nftables rules let it pass through to the normal routing table rather than forcing it into wg0. This is standard Linux policy routing: Mullvad adds a high-priority ip rule that sends marked traffic to a separate routing table, and the exclusion marks tell the kernel to skip that rule. Tailscale sets its own mark (0x80000) on all packets it originates, and all Tailscale peer addresses fall within 100.64.0.0/10 (IANA-reserved for carrier-grade NAT, reused here for the mesh network). Those two facts give us everything we need to intercept and re-mark Tailscale traffic before Mullvad\u0026rsquo;s rules process it.\nThe nftables Solution Instead of fighting both VPNs at the routing level, we inject the Mullvad exclusion marks onto Tailscale traffic before Mullvad\u0026rsquo;s own rules see it. The table uses two chains, each registered at priority -100 so they run before Mullvad\u0026rsquo;s own nftables hooks.\nThe prerouting chain handles traffic arriving from the Tailscale mesh (source in 100.64.0.0/10). The output chain handles two cases: packets Tailscale generated itself (carrying the 0x80000 mark) and packets your applications send to Tailscale addresses (destination in 100.64.0.0/10). All three paths set the same pair of exclusion marks, landing in the same bypass outcome.\nflowchart TD PKT[\u0026#34;Packet enters kernel\u0026#34;] --\u0026gt; PRE{\u0026#34;prerouting\u0026lt;br\u0026gt;src 100.64.0.0/10?\u0026#34;} PRE --\u0026gt;|Yes| MPRE[\u0026#34;ct mark 0x00000f41\u0026lt;br\u0026gt;meta mark 0x6d6f6c65\u0026#34;] PRE --\u0026gt;|No| OUT{\u0026#34;output\u0026lt;br\u0026gt;Tailscale mark 0x80000?\u0026#34;} OUT --\u0026gt;|Yes| MOUT[\u0026#34;ct mark 0x00000f41\u0026lt;br\u0026gt;meta mark 0x6d6f6c65\u0026#34;] OUT --\u0026gt;|No| DST{\u0026#34;dst 100.64.0.0/10?\u0026#34;} DST --\u0026gt;|Yes| MDST[\u0026#34;ct mark 0x00000f41\u0026lt;br\u0026gt;meta mark 0x6d6f6c65\u0026#34;] DST --\u0026gt;|No| MULLVAD[\u0026#34;Route via Mullvad wg0\u0026#34;] MPRE --\u0026gt; BYPASS[\u0026#34;Bypass Mullvad tunnel\u0026#34;] MOUT --\u0026gt; BYPASS MDST --\u0026gt; BYPASS Configuration Create The nftables Config\n# Create the nftables directory if it doesn\u0026#39;t exist sudo mkdir -p /etc/nftables # Create the config file sudo vim /etc/nftables/mullvad-tailscale.nft# /etc/nftables/mullvad-tailscale.nft table inet mullvad-tailscale { chain prerouting { type filter hook prerouting priority -100; policy accept; # Mark traffic arriving from the Tailscale mesh (100.64.0.0/10 CGNAT range) ip saddr 100.64.0.0/10 ct mark set 0x00000f41 meta mark set 0x6d6f6c65; } chain outgoing { type route hook output priority -100; policy accept; # Mark packets Tailscale originated itself (carries 0x80000 by default) meta mark 0x80000 ct mark set 0x00000f41 meta mark set 0x6d6f6c65; # Mark application traffic destined for Tailscale peers ip daddr 100.64.0.0/10 ct mark set 0x00000f41 meta mark set 0x6d6f6c65; } }\nLoad The Rules\n# Load the table immediately into the running kernel sudo nft -f /etc/nftables/mullvad-tailscale.nft\nMake Changes Persistent\nThis step is critical, systemctl enable nftables loads /etc/sysconfig/nftables.conf on Fedora, not your custom file. Without the include, the rules vanish on next boot and Tailscale silently routes through Mullvad again.\n# Tell the nftables service to also load your custom table on boot echo \u0026#39;include \u0026#34;/etc/nftables/mullvad-tailscale.nft\u0026#34;\u0026#39; | sudo tee -a /etc/sysconfig/nftables.conf # Enable and start the nftables service sudo systemctl enable --now nftables\nVerify The Table Is Loaded\n# List the table contents to confirm both chains are present sudo nft list table inet mullvad-tailscale\nAutomating Tailscale on Boot If tailscaled is already enabled, it reconnects automatically on boot using the stored node key in /var/lib/tailscale/. Check that first:\n# If this is already active, you\u0026#39;re done sudo systemctl status tailscaledIf you\u0026rsquo;re running tailscale up manually after each boot (common when pointing at a custom Headscale server), wrap it in a oneshot service that fires after both tailscaled and nftables are ready. The ordering matters: if nftables rules aren\u0026rsquo;t loaded before Tailscale connects, the first few packets to your Headscale server go through Mullvad, and the handshake may fail depending on whether your Headscale endpoint is reachable through Mullvad\u0026rsquo;s exit node.\n# Create the oneshot service sudo vim /etc/systemd/system/tailscale-up.service[Unit] Description=Tailscale up After=tailscaled.service nftables.service network-online.target Wants=network-online.target [Service] Type=oneshot ExecStart=/usr/bin/tailscale up --login-server https://your.headscale.server RemainAfterExit=yes [Install] WantedBy=multi-user.target# Enable the service sudo systemctl daemon-reload sudo systemctl enable --now tailscale-up.serviceIf your Headscale node key expires (check expiry_disabled in your Headscale config if you don\u0026rsquo;t want expiry), tailscale-up will fail silently. Add Restart=on-failure and RestartSec=30 to the [Service] block if you want automatic retry.\nTesting This should be enough. No complex configuration is needed.\nRegular traffic goes through Mullvad and Tailscale connections bypass Mullvad (which makes Headscale server access possible).\nWhen This Makes Sense This is useful if you need to access Headscale/Tailscale VPN protected services while also need to route all your traffic through Mullvad VPN.\nIt has worked for me for many months now and had no issues. So I\u0026rsquo;m sharing it with the world.\n","permalink":"https://danoss.me/en/categories/security/on-mullvad-vpn-and-linux-and-tailscale/","summary":"Setting up Mullvad VPN and Tailscale on Linux","title":"On Mullvad VPN and Linux... and Tailscale"},{"content":"\nPersonally, when choosing between Docker and Podman to run containerized applications, I choose Podman. Docker is much more popular and synonymous with containers at this point (container = Docker), but I really like that Podman is secure by default, has support for Pods, runs rootless containers by default, and doesn\u0026rsquo;t require a long-running daemon to run containers (containers are fork/exec children of the Podman process).\nA great security feature of Podman is Podman Secrets, which are similar to Kubernetes Secrets.\nSensitive Data in Environment Variables When working with containers, storing sensitive data (like password, API keys, certificates) into regular .env or configuration files is risky. These are just files protected by filesystem permissions, with sensitive values in plaintext somewhere on disk.\nWhen passing sensitive data as environment variables into the container, the values can leak in many places:\nps aux might expose process arguments with the sensitive data. podman inspect on a container shows the sensitive data in the output. If the container crashes, these data might end up in logs. To store sensitive data use Podman Secrets instead.\nWhat Are Podman Secrets? Podman Secrets is a native secret management feature in Podman that hides sensitive data from images and the filesystem. Allows containerized applications to access sensitive data (credentials, API keys, certificates) without embedding them in container images or passing them through insecure channels.\nCharacteristics\nSize Limit Secrets have a maximum size of 512KB. This size limit is perfect for credentials, certificates, and config files. You shouldn\u0026rsquo;t store media files or large data as secrets. Runtime-Only Availability Secrets exist only at container runtime in tmpfs (RAM) inside the container. They are mounted when the container starts, are available only during execution, and are unmounted when the container stops. Image Isolation Secrets cannot be embedded in images. They are never written to the container\u0026rsquo;s writable layer, included in podman commit, in container archives (podman export), or being distributed through image registries. Process Isolation Each container gets its own copy of the secret. There is no cross-container secret visibility. What Podman Secrets Protect Against Secrets leaking in container images. Secrets appearing in podman inspect output. Secrets in process listings. Secrets persisting on disk after container removal. Accidental exposure in backups of container configs. What Podman Secrets Don\u0026rsquo;t Protect Against Podman Secrets are stored as base64-encoded files in the host. They are not encrypted. Anyone with read access to your user account can decode them. They provide better security than plaintext files but can be decoded.\nPodman Secrets don\u0026rsquo;t protect against:\nA compromised container (the secret is in the container\u0026rsquo;s memory). Root access on the host system. Someone with access to your user account running podman secret inspect --showsecret. If someone has root on your system or has compromised your account, they can get your secrets, but at that point, you have bigger problems.\nSecret Storage Locations Location in the Host\n~/.local/share/containers/storage/secrets/ User-specific secret location for rootless containers. Provides no cross-user secret access. /var/lib/containers/storage/secrets/ System-wide secret location for root containers. Location Inside the Container\n/run/secrets/ Secret location inside the container. Accessed as a read-only file for the container process. Working with Podman Secrets Creating Secrets From STDIN\nCreate a secret by piping content to the command. # Create a secret by piping content directly echo -n \u0026#34;my-password\u0026#34; | podman secret create db---- \u0026lt;br\u0026gt;password -From a File\nCreate a secret from an existing file. # Create a file with secret content echo \u0026#34;my-password\u0026#34; \u0026gt; secret-file.txt # Create secret from existing file \u0026#34;secret-file.txt\u0026#34; podman secret create mysecret /path/to/secret-file.txtAdd Secrets to Containers Add a secret to a container either as a file mounted in the container\u0026rsquo;s filesystem or as an environment variable.\n# Syntax podman run --secret \u0026lt;secret\u0026gt;,[option=\u0026lt;option\u0026gt;] # options: # - type=mount - Mount secret as a file (default) # - type=env - Add secret as an environment variable # - target=\u0026lt;path\u0026gt; - Path of the mounted secret inside the container # As a file (mounted at /run/secrets/ by default) podman run --secret db-password myimage # As an environment variable podman run --secret db-password,type=env,target=DB_PASSWORD myimage # Mounted at a custom path podman run --secret db-password,target=/app/config/password myimageManaging Secrets # List all secrets podman secret ls # Inspect secret metadata (shows details and labels but doesn\u0026#39;t shows value) podman secret inspect mysecret # Actually show the secret value podman secret inspect --showsecret mysecret # Remove a secret podman secret rm mysecretExternal Secret Stores You can go a step further in secret management by using an external secret management system, to separate the secret from the host and have the container retrieve the secret at runtime from a separate authenticated system.\nSome good external secret stores are:\nHashiCorp Vault, which is feature complete and provides really good security, but might be too complex and cumbersome to manage for most cases. Bitwarden Secret Manager, which is great if you are already using Bitwarden and don\u0026rsquo;t want to use another tool. Infisical, which for me provides a good middle-ground. Really simple to setup and use, its open-source, and self-hostable. Personally, I use Infisical. In The End In the end, the choice for secret management depends on complexity and threat level. For most purposes the native Podman Secrets + good local system security is enough, but you can use an external secret store if you are working on more complex projects.\n","permalink":"https://danoss.me/en/categories/linux/podman-secrets-storing-sensitive-data-in-containers/","summary":"Using Podman Secrets for injecting sensitive data into containers","title":"Podman Secrets: Storing Sensitive Data in Containers"},{"content":"Here\u0026rsquo;s some thoughts on optimizing container images using multi-stage builds and distroless images.\nNormally, when building container images for running containerized applications, the image itself usually includes an entire OS, package manager, and system utilities, making the image size hundreds of MB to GBs.\nThis huge size for running a simple binary has an impact on deployment speed, hosting costs, and security posture.\nMulti-stage builds and distroless images optimize container image size while maintaining functionality through base image selection and removal of unnecessary runtime dependencies.\nTraditional Single-Stage Build For example, here is what most people start with when containerizing a Java application:\nFROM openjdk:11-jdk WORKDIR /app COPY . /app RUN javac Main.java CMD [\u0026#34;java\u0026#34;, \u0026#34;Main\u0026#34;]This works fine. The application runs. But the final image includes the entire JDK compiler, debugging tools, and libraries. All which is needed for development, but unnecessary at runtime.\nThe final image might end up with a size of hundreds of MB, for a compiled Java class file that might be 5KB.\nMulti-Stage Builds With multi-stage builds, you can separate build-time dependencies from runtime requirements. You compile in one stage using all the heavy tooling, and then copy only the final artifact to a minimal runtime image.\nFor example, for a containerized Java application, the build stage uses the full JDK tools and libraries, while the final imagein stage 2 uses only the JRE-slim to run the compiled code. All dev tools get left behind. This cuts image size by around 40% while keeping the same functionality as traditional single-stage builds.\n# Stage 1: Build FROM openjdk:11-jdk AS build WORKDIR /app COPY . /app RUN javac Main.java # Stage 2: Runtime FROM openjdk:11-jre-slim WORKDIR /app COPY --from=build /app/Main.class /app/ CMD [\u0026#34;java\u0026#34;, \u0026#34;Main\u0026#34;]Distroless Images Distroless images take the \u0026ldquo;image minimization\u0026rdquo; efforts a step further. For example, a distroless image might be around 2MB.\nA distroless image strips away everything except what your containerized application needs to run. No package managers, no shell, no standard Unix utilities. Just the app, its direct runtime dependencies, and minimal parts of a Linux distribution.\nFor example, for a containerized Java application using multi-stage builds with a distroless image, the resulting image size would be minimal.\n# Stage 1: Build FROM openjdk:11-jdk AS build WORKDIR /app COPY . /app RUN javac Main.java # Stage 2: Runtime FROM gcr.io/distroless/java11-debian11 WORKDIR /app COPY --from=build /app/Main.class /app/ CMD [\u0026#34;Main\u0026#34;]Important: The CMD instruction in the Dockerfile/Containerfile must use the exec form ([\u0026quot;executable\u0026quot;, \u0026quot;arg1\u0026quot;, \u0026quot;arg2]) rather than the shell form (executable arg1). Without a shell in the image, the shell form won\u0026rsquo;t work.\nAvailable Distroless Images Google and Chainguard offer distroless (base and language-specific) images, which include only CA certificates and timezone data in their base images (the bare essentials for most networked applications).\nGoogle Distroless Images\nURL: gcr.io/distroless. Google uses Debian-based images built with Bazel. Offers the most popular distroless images. Chainguard Images\nURL: images.chainguard.dev/directory. Chainguard uses Wolfi-based images built with apko/melange. Focuses on aggressive and faster CVE security patching. What\u0026rsquo;s Not Available Complex applications like databases are not available as distroless images. For example, they require configuration management tools, backup and restore tools, scripts, etc. Too much for a minimal image.\nSome things not available in distroless form are:\nDatabases: PostgreSQL, MySQL, MongoDB need backup tools, admin CLIs, and configuration utilities. Web Servers: Nginx, Apache require configuration management and reload capabilities. Others: Redis, Ruby, PHP (no official images available). The Debugging Challenge The main tradeoff with distroless images is debugging. No shell means you can\u0026rsquo;t docker exec into a running container to investigate issues. By design, you don\u0026rsquo;t want don\u0026rsquo;t want those tools in production, but it requires different debugging approaches.\nDebug Image Variants Google provides debug variants of their distroless images with busybox included to test and debug where you need shell access.\nFROM gcr.io/distroless/java11-debian11:debugOnce the debug is finished remove them. The whole point of distroless is minimizing the attack surface.\nNamespace Sharing Technique You can run a 2nd busybox container that share namespaces (PID and network) with your distroless container for debug purposes only.\nThis allows tools like ps and ss/netstat to show the same processes and connections as the target container.\n# Start distroless container docker run -d --name myapp gcr.io/distroless/static-debian12 /app # Attach a debugger container docker run --rm -it \\ --name debugger \\ --pid container:myapp \\ --network container:myapp \\ busybox shThis approach is lightweight and doesn\u0026rsquo;t require restarting your application container.\nKubernetes Ephemeral Containers If you\u0026rsquo;re running on Kubernetes, ephemeral containers provide built-in debugging:\nkubectl debug -it \u0026lt;pod_name\u0026gt; \\ --image=alpine \\ --target=\u0026lt;container_name\u0026gt;This creates a temporary debugging container sharing namespaces with the target pod. Similar to regular namespace sharing but integrated into Kubernetes.\nWhy Image Size Matters When you build a container image the traditional way, everything used during the build process ends up in the final image.\nFor example, if you need a compiler to build app, it will be included in the image. Package managers for installing dependencies? They\u0026rsquo;re there too. And everything will be in the final image in production.\nThis bloat creates some problems:\nDeployment Speed: A 400MB image takes much longer to pull than a 10MB one. When scaling horizontally or deploying to multiple hosts, the extra seconds add up quickly. Infrastructure Costs: Container registries charge based on storage and bandwidth. A 400MB image stored across 10 versions costs more money. Security Surface Area: Every binary, library, and package is a potential vulnerability. CVEs can impact components you never use but happen to exist in the image. Smaller images mean fewer components to patch. Resource efficiency: Smaller images mean faster starts, less memory pressure, and more efficient disk I/O. With a small image size, you are not only saving storage space costs, but also improving your security posture, and making application deployment faster.\nDistroless, Alpine, or Standard Images? Alpine is a lightweight Linux distribution. Its small size can rival distroless and could be a better choice than distroless when building a minimal container image.\nThe choice between distroless, Alpine, and standard minimal images isn\u0026rsquo;t about one being \u0026ldquo;better\u0026rdquo;, as everything in life, it depends on your use case.\nSome comparison information:\nAspect Distroless Alpine Standard Minimal (debian-slim) Image Size Smallest (static: ~2MB, Java: ~40-120MB) Very small (base: ~5MB, Java: ~80-150MB) Small (base: ~25MB, Java: ~200-250MB) Security Minimal attack surface, no shell Small surface, includes shell/package manager Larger surface, full userland tools Debugging Difficult (no shell, no utilities) Easy (shell, standard Unix tools) Easy (full toolset available) CVE Count Lowest (minimal packages) Low (small package set) Higher (more packages installed) Compatibility Standard glibc musl libc (can cause compatibility issues) Standard glibc, widest compatibility Use Case Production, security-first Development, debugging, general use Legacy apps, maximum compatibility When To Use Distroless Production Environments: Where security and efficiency is more important than debugging convenience. You usually have good logging / observability. Running Statically-Linked Applications: When running Go and Rust programs with no dependencies. No operating system or packaging systems are needed. Security is Top Priority: Where security is top of mind and reducing the attack surface and isolation is very important. When To Use Alpine Development Environments: Where you need quick iteration and the ability to install tools on the fly via its package manager. Debugging Needed: When you need to troubleshoot issues by exec\u0026rsquo;ing into containers. Alpine provides a shell and common utilities without excessive bloat. Runtime Tooling Required: Applications that need curl, wget, nc, or other utilities at runtime. Don\u0026rsquo;t Mind Compatibility: When your app doesn\u0026rsquo;t depend on glibc-specific features. Most apps work fine with musl libc. C/C++ applications compiled against glibc may have issues. When To Use Standard Minimal Images Use standard minimal images like debian-slim or ubuntu-minimal when:\nMaximum Compatibility: Legacy applications that expect specific system libraries or glibc behavior. Mixed Workloads: When running multiple applications with different requirements in the same cluster. Standardizing on Debian-slim can simplify operations even if it\u0026rsquo;s not optimal for every service. Best Practices A few things I\u0026rsquo;ve learned working with distroless images:\nAlways Use Multi-Stage Builds: Even if you\u0026rsquo;re not using a distroless image. The separation between build and runtime environments is good practice regardless. Pin Your Base Image Versions: Use tags or SHA256 digests rather than latest to use a specific image version. This ensures reproducible builds and makes rollbacks predictable. Scan Your Images Regularly: Scan your container images regularly for vulnerabilities. In The End Multi-stage builds and distroless images are a simple and really good solution to not only reduce the image size of container images, but also provide better security and speed, which becomes increasingly important as you scale. The key is understanding the tradeoffs and why would you choose a normal build, distroless, a minimal image, or alpine.\n","permalink":"https://danoss.me/en/categories/linux/container-image-optimization-multi-stage-builds-and-distroless-images/","summary":"Thoughts on optimizing container images using multi-stage builds and distroless images.","title":"Container Image Optimization: Multi Stage Builds and Distroless Images"},{"content":"\nContrary to popular belief, sometimes making yourself invisible in the web (and even IRL) makes you stand out.\nThere are tons of techniques that bad actors and businesses/orgs do to deanonymize you and track you, one of them is browser fingerprinting.\nWhat is Browser Fingerprinting? Browser fingerprinting is technique that makes it difficult to maintain privacy on the web.\nThis technique works by collecting and analyzing different unique characteristics of the user\u0026rsquo;s browser and system configuration. This is very successful in identifying individual users because a person that turns on multiple browser extensions can be identified across multiple websites due to this not so common behavior.\nHere are Some Common Fingerprinting Data Points:\nScreen resolution and color depth. Installed fonts and system information. Timezone and language settings. Browser version and plugins. Canvas rendering (how the browser draws graphics). WebGL (GPU characteristics). Audio processing (how your device handles sound). Hardware specifications (CPU, RAM). All this data combined with other user tracking techniques like cookies, active fingerprinting make it easier for 3rd parties to identify who you are in the web.\nPrivacy Efforts Make You Unique Interestingly, doing the effort of protecting your privacy on the web by installing privacy-focused extensions, and others will make you more unique among the rest of regular users. The uniqueness itself becomes the identifying factor.\nFor Example: If you\u0026rsquo;re one of millions of users with a 1920×1080 screen running Firefox on Windows 11, you blend into the crowd. But you will be a unique user if you\u0026rsquo;re reporting a randomized 1337×768 screen with UTC timezone. This is like trying to hide on the crowd wearing a neon hoodie with a mask. You will stand out from the crowd.\nThe Goal of Browser Fingerprinting Trackers and businesses don\u0026rsquo;t try to deanonymize you or get your personal information, but instead to track and link your browsing behavior across different websites.\nFor Example:\nYou visit a shoe store (tracker sees fingerprint 12345). You visit a coffee shop (tracker sees fingerprint 12345). The tracker links both visits and can create a digital profile that says: \u0026ldquo;user fingerprint 12345 likes shoes AND coffee\u0026rdquo;. Now, next time the user with fingerprint 12345 searches for shoes, they will also ge getting coffee ads. Fingerprinting Techniques There are many fingerprinting techniques, but the most common ones are:\nCanvas Fingerprinting: When accessing any website, your browser draws text and shapes on a hidden canvas. Different GPUs, operating systems, and fonts render these slightly differently, allowing trackers to uniquely identify you based on the canvas.\nWebGL Fingerprinting: This is similar to canvas fingerpritning but with 3D graphics rendering. Your GPU and driver version can create a unique identify of you.\nAudio Fingerprinting: Hardware devices process audio slightly differently due to hardware variations and audio processing algorithms, etc. This creates another unique identifier.\nFont Enumeration: Websites can detect installed fonts on your system. A unique combination of fonts can identify you.\nCSS-based Fingerprinting: New techniques using CSS container queries, @supports, and @import can fingerprint browsers without JavaScript.\nBehavior Fingerprinting: Some user patterns can expose users, like tracking mouse movements, typing patterns, and scrolling behavior.\nHow To Protect Yourself? There are 2 approaches to protect yourself.\nApproach 1: Make Everyone Identical Privacy-focused browsers like Mullvad, Librewolf, and the Tor browser make the effort to introduce privacy-focused and secure defaults.\nThe recommendation with this approach is for users to not change the default values (which are already tuned for privacy and security) to make every user present an identical fingerprint (same screen size, timezone (UTC), fonts, and user agent).\nThis makes every user look the same, but websites can easily detect you are using that browser and may deny access if they don\u0026rsquo;t want these users to access the website.\nApproach 2: Randomize Data Per-Site Browsers like Brave and Firefox can randomize fingerprint data per site.\nInstead of making everyone look the same, randomize your fingerprint data on each website to break tracking across sites. For example, when you visit Amazon, your browser reports one set of characteristics, and when you visit Facebook, it reports different ones.\nThis makes is harder for trackers to link your browsing activity across sites, but you can still be tracked as someone using anti-fingerprinting techniques, so the success is questionable.\nPractical Defense Strategy Here are some things you can do to minimize the fingerprinting.\nUse an Ad Blocker Install uBlock Origin.\nHonestly, this is the best browser extension you can use.\nThe default installation should work fine for most users, provides great protection, and blocks ads. However, if you do an advanced configuration, you can block scripts, individual trackers, elements in websites, fonts, etc. Always use this.\nUse a Privacy-Friendly Browser Do NOT use Google Chrome, Edge, or Safari. I don\u0026rsquo;t recommend the Tor browser for regular browser usage either.\nI recommend these browsers with their appropriate configuration (check the docs for each)\nFirefox. A privacy-friendly Firefox fork like Librewolf. Brave browser. Don\u0026rsquo;t Manually Spoof Values For most users, manually spoofing timezone, screen size, or user agents will make them more unique. Just let the privacy-friendly browser do its job.\nThe Sad Truth Avoiding fingerprinting is hard (if not impossible). Even with all precautions, trackers with enough resources can likely identify you.\nFingerprinting techniques are difficult to avoid or spoof because they rely on the actual hardware you use and trying to protect your privacy by making privacy-friendly changes to your browser will make you stand out from the crowd.\nEven worse, as of February 16, 2025, Google began allowing advertisers using its ads platform to use fingerprinting techniques, making it a more challenging environment for privacy-friendly people (source).\nIn The End Even if fingerprinting is hard to avoid, I still recommend it because:\nThe ad-blocker will block ads on websites (without it, your Internet browsing experience is reduces 100x, some blogs and news websites are ridden with ads). Blocks bad actors and shady websites from creating a profile of you. Prevents user cross-link between some websites and reducing the amount of personalized ads and information about you from being public to some of these trackers/businesses. Additional Resources Fingerprinting Tests\nCover Your Tracks - EFF\u0026rsquo;s fingerprinting test using real tracking companies PrivacyTests.org - Regular tests run against several browsers, including LibreWolf BrowserLeaks - All-in-one browser testing tool Further Reading\nFirefox\u0026rsquo;s Resist Fingerprinting - Technical details on how Firefox handles fingerprinting Brave\u0026rsquo;s Fingerprinting Protection - How Brave approaches the problem W3C Fingerprinting Guidance - Browser standards perspective on fingerprinting AmIUnique Research - Academic papers on fingerprinting ","permalink":"https://danoss.me/en/categories/security/what-is-browser-fingerprinting/","summary":"What is browser fingerprinting and how to protect yourself","title":"What is Browser Fingerprinting?"},{"content":"\nThinking about the latest anti-privacy trend, I decided to organize my thoughts about digital footprint and what to do about it.\nYou leave traces every time you use the Internet (download an image, see a video, etc.). This does not only reveal your browsing history, but also allows people and businesses to create a digital profile of you.\nThe digital profile can reveal who you are, where you go regularly, who are your friends and relatives, what you like, and predict where you will be.\nThis data gathering is all over the place, so even if you\u0026rsquo;re privacy-conscious, your digital footprint is there and is very difficult to hide.\nDigital Profiling Digital profiling is \u0026ldquo;the process of gathering and analyzing information about an individual that exists online. A digital profile can include information about personal characteristics, behaviors, affiliations, connections and interactions\u0026rdquo; (source).\nThese profiles of you are being created by:\nCompanies: Google, Meta, Amazon, and others collect data to serve targeted ads and improve their services. Data Brokers: Companies that buy and sell data about you to advertisers, insurance companies, and others. Government Agencies: Law enforcement and intelligence agencies collect and analyze data for surveillance and investigations. Bad Actors: Bad actors, criminals, cartels, you name it. can create digital profiles of their victims using publicly available information. How Profiling Works Connect various \u0026ldquo;unrelated\u0026rdquo; pieces of data of an individual to create a digital profile.\nFor Example: If you post a photo to Instagram from a coffee shop, you can reveal:\nLocation: GPS coordinates embedded in the photo metadata can reveal your location. Time: Metadata can reveal when you were there (you can even check the shadows of the sun on objects to figure out the hour/day). Friends: Who you were with (you can figure out the people in the photo if you tag them or if their face is visible). Preferences: The type of coffee you like or type of shop you frequent. Patterns: If you post photos from the same coffee shop multiple times over the days/months, someone might infer that it\u0026rsquo;s near your home or workplace. This example was with just a simple photo, but businesses and individuals that specialize on data collection and analysis can do much more, like analyzing data gathered from websites you visit, login sessions, purchases, Wi-Fi networks you connect to, usernames, emails, your phone number, your IP (if not using VPN), phone IMEI, browser fingerprints, etc.\nAfter gathering all this information and creating a digital profile of you, data brokers sell this information to 3rd parties.\nThe Shadow Profile Problem A major problem that will not go away is shadow profiles.\nDefinition: A shadow profile is a set of data collected about someone without their explicit consent (source).\nWhat is more dangerous, even if you delete your accounts or never sign up, some platforms still build profiles without your consent:\nFacebook Shadow Profiles: Facebook creates profiles for non-users by extracting contact information from users who upload their address books. Data Persistence: When you \u0026ldquo;delete\u0026rdquo; content from social media, it\u0026rsquo;s often only hidden from public view. The data remains on company servers for backup, legal, or analysis purposes (believe some companies retain this data for 3, 5, or 10 years for this reason). 3rd-Party Tracking: Advertising networks and analytics services track you across websites even without account logins through cookies, browser fingerprints, and tracking pixels. Nothing to Hide, What is the Risk? The \u0026ldquo;nothing to hide\u0026rdquo; argument is done by ignorant people that are oblivious or misunderstand the risks.\nPrivacy is a right, people act differently when they know they are being watched. No privacy means businesses, corrupt governments, and bad actors can create digital profiles of individuals and take action against them, such as:\nTargeted Attacks Detailed profiles enable targeted attacks:\nSpear Phishing: Attackers use publicly available information to create convincing phishing emails referencing real info about your life, work, or relationships. Social Engineering: Bad actors can impersonate your friends or yourself if they know your patterns, contacts, and preferences. Physical Security: Posting vacation photos in real-time shows that your home is empty, allowing bad actors to take advantage. Stalking and Harassment: Aggregated location and social data enables stalkers to track movements and predict future locations. Association by Proximity: Being friends with or related to someone of interest to law enforcement or bad actors makes you a data point in their profile. Beware of who you are friends with. No More Privacy Once data exists online, anyone (with some effort) can use it:\nData Breaches: Companies holding your data get breached regularly. Your digital profile will end up in the hands of criminals or anyone targeting you. Terms of Service Changes: Some companies change privacy policies regularly. Data you shared under one policy may be used differently later. Corporate Acquisitions: When companies are bought, your data is part of the acquisition. New owners may have different views on privacy. Government Access: Law enforcement can request data from companies and can gather data themselves. Discrimination and Manipulation Profiles can be used against you in ways you never consented to:\nTargeted Pricing: Companies use profiles to show different prices to different people based on predicted willingness to pay. Insurance and Employment: Some insurers and employers use data broker information to assess risk or screen candidates. Political Manipulation: Profiling enable political advertising designed to manipulate emotions and influence voting behavior. Reduce Your Digital Footprint Perfect privacy is nearly impossible, but you can reduce unnecessary data collection and make profiling more difficult. The idea is to create a balance between your own privacy efforts and ease of use. Perfect privacy can be painful in everyday life.\nSeparate Your Digital Identities Don\u0026rsquo;t use the same username, email, or profile across all services:\nUse Different Usernames: Avoid using the same handle across platforms. This makes correlation harder. Create Context-Specific Email Addresses: Use separate emails for different purposes (shopping, social media, work, banking). Email Aliasing Services: Tools like SimpleLogin, AnonAddy let you create unique addresses for each service that forward to your main inbox. Different Profile Pictures: Don\u0026rsquo;t use the same photo across platforms. Reverse image search makes correlation trivial otherwise. Strip Metadata from Photos Before uploading images anywhere use metadata removal tools like exiftool to remove its metadata.\nNote: Most social media platforms strip EXIF data automatically, but this happens on the server-side, they see the metadata before removing it. Strip it yourself before uploading.\nMinimize Location Sharing Disable Location Services: Turn off location access for apps that don\u0026rsquo;t need it. Disable Location History: Disable Google Timeline, Apple\u0026rsquo;s Significant Locations, and similar features from similar apps. Wi-Fi and Bluetooth Scanning: Disable \u0026ldquo;Wi-Fi Scanning\u0026rdquo; and \u0026ldquo;Bluetooth Scanning\u0026rdquo; in location settings. Use Different Payment Methods Break the payment trail:\nVirtual Card Numbers: Services like Privacy.com, your bank\u0026rsquo;s virtual card feature, or credit cards that generate single-use numbers prevent merchants from sharing your real card number. Separate Cards for Different Uses: One card for online purchases, another for recurring subscriptions, another for in-person transactions. Crypto: Use Monero or similar privacy-friendly cryptocurrencies for payment. Avoid Linking Everything to One Account: Don\u0026rsquo;t link every service to the same PayPal or payment account. Review and Minimize Social Media Social media is the easiest place to leak information:\nReview Old Posts: Search your own name and usernames periodically. You might be surprised what\u0026rsquo;s still public. Limit Tagging: Configure privacy settings to approve tags before they appear on your profile. Don\u0026rsquo;t Post Real-Time Locations: Share vacation photos after you return, not while you\u0026rsquo;re away. Separate Personal and Professional: Use different accounts for personal life versus professional presence. Use Privacy-Focused Tools Where possible, choose services that don\u0026rsquo;t monetize your data:\nBrowsers: Firefox with privacy extensions (uBlock Origin, Privacy Badger), or Brave. Search Engines: Use DuckDuckGo or Brave Search instead of Google. DNS: Use encrypted DNS to prevent your ISP from logging every domain you visit. VPN: Always use a trusted zero-knowledge VPN like Proton or Mullvad to browse the web. A VPN hides your IP address from websites and your ISP (but the VPN provider can see your traffic). Password Managers: Use unique passwords for every service. If one gets breached, others remain secure. OnePassword, Bitwarden and others are great. DO NOT USE LASTPASS. Final Thoughts Everyone has been digital profiling you for years. The best you can do to protect yourself is to make informed decisions on what data you\u0026rsquo;re comfortable sharing and take reasonable steps to limit unnecessary exposure.\nStart protecting yourself by making a few changes that make sense for your threat model and lifestyle, then build from there for maximum privacy.\nFurther Reading:\nElectronic Frontier Foundation - Surveillance Self-Defense Privacy Guides The New Oil - Privacy \u0026amp; Security Resources ","permalink":"https://danoss.me/en/categories/security/your-digital-footprint/","summary":"Thoughts on your digital footprint and what to do about it","title":"Your Digital Footprint"},{"content":"With Windows 11\u0026rsquo;s privacy concerns and Microsoft\u0026rsquo;s bad decisions overall (web-based system apps consuming GBs of RAM for example), many people are looking to move to Linux.\nI want to share some thoughts on this process as there are many things to consider.\nRemember: Always test your Linux distribution of choice with a Live USB.\nHardware Compatibility Verify your hardware actually works in Linux, for example:\nWiFi and Network Adapters: Some WiFi and network adapters require additional drivers or simply don\u0026rsquo;t work as it should. Boot a live USB and make sure you can connect to your network. NVIDIA GPUs: The proprietary drivers work fine these days (sometimes). The open-source nouveau drivers exist but have limited performance. If you game or do GPU-intensive work, check that your card is supported by the current NVIDIA driver packages or change to AMD. Peripherals and Function Keys: Now, most keyboards and mice work out of the box, but special features or FN keys might not. In my case, my Logitech mouse worked fine, but special configurations required Logitech\u0026rsquo;s Windows software (which sucks). I worked around this by remapping mouse gestures to achieve similar functionality, which is not perfect, but good enough. Touchpads, fingerprint readers, and webcams usually work, but check with a Live USB before you commit. Software Alternatives Most Windows applications won\u0026rsquo;t run on Linux. That\u0026rsquo;s just how it is.\nMS Office: If you use Microsoft Office (Word, Excel, PowerPoint) heavily, the alternatives might not be good enough for you. LibreOffice and OnlyOffice are solid, but if you depend on complex Excel macros or specific formatting in Word documents, you\u0026rsquo;ll might not like the change to Linux. The web versions of Office 365 work fine, but they\u0026rsquo;re limited compared to the desktop apps. Creative Software: Adobe products don\u0026rsquo;t run natively on Linux. GIMP exists as a Photoshop alternative, but it\u0026rsquo;s a different tool with a different workflow. Gaming: This has improved dramatically thanks to Valve\u0026rsquo;s Proton. Steam works great and most single-player games run well. Check ProtonDB for your specific games. Now, multiplayer games with kernel-level anti-cheat (like Battlefield), won\u0026rsquo;t work on Linux because their anti-cheat systems won\u0026rsquo;t run outside of Windows. Specialized Software: CAD programs, industry-specific tools, certain VPN clients, etc. won\u0026rsquo;t work on Linux or don\u0026rsquo;t have an alternative app. WINE can run some Windows applications, but might break with updates. Choosing Your Distribution Honestly, there are too many Linux distributions (which is good due to freedom of choice and bad because there is just too many), but the recommendation is to use something stable and with good community support.\nMy top recommendations are:\nFedora: This is what I use, cutting-edge packages, great defaults, secure, and stable enough for daily use. The 6-month release cycle means you get new software at a good pace. Ubuntu/Debian: They have the most documentation and largest package repositories. I tend to recommend Ubuntu less because of their really dumb choices the past few years like the Unity DE, snap, pro, etc, but it\u0026rsquo;s still good for new folks. Arch/Manjaro: The meme. They have rolling releases with bleeding-edge packages, but require more maintenance and troubleshooting. Here is an article for popular Linux distros: https://www.geeksforgeeks.org/linux-unix/8-most-popular-linux-distributions/\nPackage Management Forget downloading .exe installers from random websites. Linux uses centralized package managers:\ndnf on Fedora/RHEL. apt on Debian/Ubuntu. pacman on Arch-based systems. Everything installs through your package manager. Updates are centralized and are much safer than the Windows model of downloading installers from wherever and hoping they don\u0026rsquo;thave malware.\nBesides traditional packages, there is also AppImages (portable applications) and Flatpaks (sandboxed apps with their own dependencies). Each has trade-offs, but they are good options and easy to install.\nBlog posts with more info on Flatpaks:\nWhy Flatpaks Take So Much Storage Space? Flatpak Stopped Working Desktop Environments Linux offers multiple desktop environments, you\u0026rsquo;re not locked into one interface like Windows.\nGNOME (Default in many distros) provides a modern and simple workflow. It works well but doesn\u0026rsquo;t offer many features unless you also install the Tweaks app and additional GNOME extensions. Not perfect but really good if you get used to it. KDE Plasma is highly customizable and feels more Windows-like if configured that way. COSMIC is System76\u0026rsquo;s desktop environment written in Rust. Really good and intuitive but is still pretty new with some fixes and tweaks needed. Really promising stuff. Hyprland is a modern Wayland compositor with dynamic tiling. Popular with the ricing and meme community. It\u0026rsquo;s really fast and customizable. XFCE/LXQt are lightweight options that run well on older hardware. Each desktop environment has different resource requirements and philosophies. Try them with live USBs before deciding.\nMaking the Decision Plan Your Migration Don\u0026rsquo;t just wipe Windows immediately.\nBack up everything first: documents, browser profiles, email configurations. Export anything tied to Windows-specific tools.\nI don\u0026rsquo;t recommend dual-booting because it\u0026rsquo;s a half-measure. First test if you like Linux and if everything works as it should with a VM (virtual machine) and/or Live USB.\nWhy Choose Linux Linux gives you total control on what happens on your system.\nYou don\u0026rsquo;t really need to be super technical to use it. Nowadays there is a graphical application for almost everything.\nLinux gives you more privacy, knowledge, and freedom that Windows restricts. If you care about these principles then Linux is for you.\nWhy Stay in Windows At work, most orgs use AD (Active Directory), MS Office, proprietary software, etc. all of which run on Windows.\nIf a specific software or hardware tool you purchased runs on Windows or has many problems in Linux, you might want to stay in Windows.\nIf you can\u0026rsquo;t live without a specific multiplayer game with kernel-level anti-cheat at the Windows kernel level, you might want to stay in Windows (check ProtonDB for compatibility).\nEach person has their own use case, choose wisely.\nMy Experience I switched to Fedora a few years ago and haven\u0026rsquo;t looked back. The initial learning was exciting and painless, but after learning how Linux works, I was able to get full control of my system. Windows nowadays feels clunky and restrictive when I need to use it.\nThe command line access, native container support, and full control over my system makes Linux a good choice for me, but this use case varies from person to person.\n","permalink":"https://danoss.me/en/categories/linux/moving-from-windows-to-linux/","summary":"Things to consider when moving from Windows to Linux","title":"Moving From Windows to Linux"},{"content":"Australia recently passed legislation banning social media access for children under 16. While \u0026ldquo;protecting children\u0026rdquo; sounds a good thing for us (the non-pedophiles), this creates serious privacy concerns that affect everyone.\nMandatory ID Verification? Social media sites like Youtube, Bluesky, Facebook must implement age verification systems. Some methods being considered involve providing:\nGovernment ID / Passport information Credit card information Social security numbers Biometric information Now, most platforms will outsource this implementation to 3rd party services. While these 3rd party services have established processes in place and more security on their data, your personal data is now shared with 2 entities and they can do whatever they want with it like sell it to the highest bidder, or other shady deals.\nPrivacy Impact\nOnce you submit your personal information for age verification to access these sites:\nYour data becomes permanent on the web. A breach on the social media website or 3rd party verification service will expose all your personal data to bad actors. Shared databases of personal information can be shared across multiple businesses and governments. No privacy for anyone. You are creating a permanent digital profile. Every website will know exactly who you are, that you are accessing their services, what you post, what your likes are, etc. This is kept forever. The Main Goal: Control This isn\u0026rsquo;t really about protecting kids (I mean, parents should take this responsibility).\nThe goal is to establish a mass surveillance infrastructure and control of the population.\nThis creates a precedent and normalization for mandatory ID verification for accessing most of Internet services. Creates a centralized database of all citizens across the globe (not just Australia). Allows governments to monitor and control the free speech of their citizens. If you are dissatisfied with the government or a government official you can be targeted and eliminated in the future. Other governments will use this kind of legislation to \u0026ldquo;protect the kids\u0026rdquo; to implement these monitoring tools because the saw that it worked in Australia. More censorship and control. What to Do? Why live in this dystopia?\nReactionary Actions\nIf legislators in your country propose similar laws, make your opposition known. Vote against these fools. These officials are either:\nNaively unaware of the surveillance infrastructure they\u0026rsquo;re building (which means they are not prepared to govern). Aware and perfectly fine with restricting citizen freedoms to gain or consolidate power. Pushing policies written by lobbyists who benefit from data collection. Proactive Actions\nThis kind of legislation will be normalized and will spread to all countries. Here are some things to do now:\nCheck your Digital Presence: Search for yourself. Check what information appears in domain registrations, old social media posts, and public databases and delete most of what you can (businesses that comply with GDPR have this option). Clean up Social Media: Delete old posts that reveal personal information. Tighten privacy settings. Do you need that 10-year-old Facebook account at all? Strip Metadata from Photos: Before posting images online, use tools to remove EXIF data (like exiftool on Linux). While most major social media platforms strip EXIF data on upload, not all sites do. Be Mindful of What you Share: Before posting, ask: \u0026ldquo;What could someone learn about me from this?\u0026rdquo; Location tags, purchase photos, vacation updates, they all create data points someone can exploit. Use Separate Identities. You can keep different personas for different activities. Your professional presence doesn\u0026rsquo;t need to connect to your hobby forums. Use different usernames, emails (email aliases are great!), profile pictures, etc. Assume Everything is Permanent: Even if you delete something, assume someone archived it. The Internet Archive, Google, and random scrapers capture more than you think. Use a VPN: Always use a VPN, this is a must. Choose a paid VPN service like Proton or Mullvad. Avoid free VPNs as they are free because they spy on your traffic. Use Encryption: Always use encrypted communication tools and become used to them. Self-Hosting: Consider self-hosting your services. While this requires some technical expertise, is not impossible to do and you can learn a lot. The Endgame This path leads to a sanitized internet where every action is tied to your verified identity. Dissent and asking incorrect questions become dangerous. Privacy becomes impossible. Free speech becomes impossible. Freedom of association becomes impossible. Freedom of movement becomes impossible. Freedom of thought becomes impossible\u0026hellip;\nMost kids are really unlikable huh?\n","permalink":"https://danoss.me/en/categories/others/think-of-the-children/","summary":"The same old story, politicians deceiving people by eliminating freedom for citizens with the excuse of protecting children","title":"Think of the Children"},{"content":"\nFrom time to time I see people that copy commands online get elevated privileges by typing sudo su.\nNow, even if it works for the intended purpose, you can probably do something better.\nUnder The Hood When you run sudo su you are chaining 2 privilege escalation commands:\nsudo executes a command as another user (root by default). It checks the /etc/sudoers config file to check if the user executing the command has permissions to do so, logs the sudo action made, and asks for the user password (if configured with the defaults). su switches to another user (root by default). It requires the target user\u0026rsquo;s password (root password) and starts a new shell session as this user. With sudo su you are using sudo to run the su command with root privileges, which then starts a root privileged shell, which is redundant.\nNow, if you need to run many privileged commands in a row, using sudo su might be tempting (nobody wants to type sudo for each and every command), but instead, you can use sudo -i, which is a much better.\nThe Problem with sudo su Limited Audit Trail\nsudo su (and sudo -i) share the same logging limitation: sudo only logs the invocation of the root shell, not the individual commands you run within it.\n# This gets logged: sudo -i # These DO NOT get logged by sudo: apt update rm -rf /important/data systemctl restart nginxThis is why sudo \u0026lt;command\u0026gt; is preferred when possible, as each individual command gets its own audit log entry.\nIn the future if you were using sudo su and something happened, you will have no idea what happened inside this root shell session. Every command executed in that su shell is invisible to sudo\u0026rsquo;s logging.\nEnvironment Variable Chaos\nsu by itself doesn\u0026rsquo;t fully reset your shell environment, it changes some environment variables (HOME, USER, SHELL, LOGNAME) but preserves others from your original user (like PATH, which may still point to your current user\u0026rsquo;s PATH).\nThis mixed environment can cause unexpected behavior on scripts.\nEnvironment Variable Handling:\nsu - preserves user environment (can cause issues) su - - provides clean root login environment sudo su - mixed environment (depends on sudo config) sudo -i - clean root login environment. Using sudo -i ensures consistent, predictable environment behavior. It\u0026rsquo;s Redundant\nsudo already gives you elevated privileges securely, why complicate things and add redundancy?\nUse sudo -i Instead If you need a persistent root shell for multiple commands, use sudo -i.\nThe -i (or --login) option starts a login shell as root with a clean environment.\nBenefits Over sudo su Clean Environment: You get root\u0026rsquo;s full login environment ($PATH, $HOME=/root, environment variables) as if you logged in directly as root. Simpler: One command instead of chaining 2 privilege escalation commands. Clearer Intent: More obviously requests a root login shell. Uses Your Password: Authenticated via sudo (your password), not root\u0026rsquo;s password. Respects sudo Policies: Access controlled by /etc/sudoers configuration. What Gets Logged sudo -i logs the invocation of the root shell, but NOT individual commands you run within that shell:\n# Logged by sudo Nov 20 10:23:45 hostname sudo: username : TTY=pts/0 ; PWD=/home/username ; USER=root ; COMMAND=/bin/bash -i # NOT logged by sudo: anything you type after thisFor comprehensive command auditing, you can use auditd, psacct, or another centralized logging solution.\nBest Practice Hierarchy Preferred: sudo \u0026lt;command\u0026gt; for individual commands Each command is logged separately Clear audit trail When Necessary: sudo -i for persistent root shell Multiple commands in a row requiring root Interactive troubleshooting System maintenance tasks Avoid: sudo su Redundant privilege escalation Same logging limitations as sudo -i Why Use sudo su? If you\u0026rsquo;re on a system where you don\u0026rsquo;t have sudo access but know the root password, then yes, su - or su -l makes sense. But if you\u0026rsquo;re reaching for sudo su, you already have sudo configured, right? Use it properly brother.\nBottom line, stop using sudo su. Use sudo -i when you need a root shell. Better yet, use sudo with individual commands.\n","permalink":"https://danoss.me/en/categories/linux/sudo-su-why-embrace-redundancy/","summary":"What is sudo su and why is it redundant? What to use instead?","title":"sudo su - Why Embrace Redundancy?"},{"content":"When you visit a website like www.google.com, your browser sends a connection request from your computer to the web server. The server processes your request and sends back the web page you requested to visit.\nThis is a pretty simplified description of what happens when you visit a website, but the point is that these direct connections expose your privacy as the web server can see your IP address / location and knows you connected from there.\nWith proxychains, you can route connection attempts to a server through a number of proxies that will hide your IP from the destination server itself. The server will only see that the proxy server attempted to connect, not you.\nWhat is Proxychains? Proxychains allows dynamically linked applications to transparently route its TCP network traffic through a chain of proxy servers so that the destination sees the request as coming from the proxy server rather than your real IP address.\nOnce you understand simple concepts like how proxychains works, proxy types, chain modes, and configuration you should have no issues with the tool. It should make sense why it doesn\u0026rsquo;t work as straightforward in some apps and how to troubleshoot.\nOfficial Repository: https://github.com/rofl0r/proxychains-ng\nHow Proxychains Works Proxychains operates as a preloaded shared library that intercepts network-related system calls in dynamically linked programs, enabling transparent proxy redirection at the application layer.\nPreloaded Shared Library: Library that contains reusable code that is injected into other program\u0026rsquo;s memory space before other libraries load, allowing it to override standard functions. Dynamically Linked Program: Programs (most of them actually), that use external shared libraries (like .so files) to perform common functions rather than bundling all code into the binary itself. In simpler words, proxychains intercept networking functions of applications to inject its proxy routing logic, all without the application know about it.\nLibrary Injection and Function Interception When an app is executed, proxychains injects the libproxychains.so library into the app process\u0026rsquo;s memory space before standard system libraries load.\nThis allows proxychains to intercept and replace networking functions from libc to add proxy redirection logic before establishing TCP connections.\nExample libc functions intercepted:\nconnect(): Captures outbound TCP connection attempts and redirects them through the proxy chain. getaddrinfo(): Handles DNS resolution requests via the POSIX standard API. close() / close_range(): Ensures proper socket and resource cleanup. Traffic Redirection Flow Once a connection attempt is intercepted, proxychains transparently reroutes the traffic through the configured proxy sequence (from proxychains.conf).\nThe application is completely unaware of the proxychains rerouting and continues to operate as usual. No change in the app is needed.\nDNS Proxying and Leak Prevention When proxy_dns is enabled in proxychains.conf, proxychains prevents DNS leaks by handling hostname resolution through the proxy chain rather than your local DNS resolver. This prevents DNS leaks that could expose your browsing activity to your ISP (internet service provider) or local network.\nWhat\u0026rsquo;s a DNS leak? When you visit a website, your computer needs to translate the domain name (like www.google.com) into an IP address through DNS resolution. Even if all your traffic is going through proxies, those DNS queries might still be sent directly to your ISP\u0026rsquo;s DNS server (revealing which sites you\u0026rsquo;re trying to visit). DNS proxying fixes this by routing those queries through your proxy chain as well.\nProtocol Support Proxychains works with TCP-based traffic and supports several common proxy types:\nTCP Connections: Full support for all TCP-based network traffic. Proxy Types: SOCKS4, SOCKS4a, SOCKS5 (with or without authentication), and HTTP/HTTPS proxies. Proxy Types SOCKS4\nThe basic SOCKS protocol that provides basic TCP proxying. Only supports IPv4 addresses and doesn\u0026rsquo;t handle DNS resolution (needs to resolve hostnames to IP addresses before connecting). Widely supported but limited compared to newer versions. Not recommended. SOCKS4a\nAn extension of SOCKS4 that adds hostname resolution support. Instead of requiring you to resolve domain names yourself, it lets the proxy server handle DNS lookups. This is useful for accessing resources by hostname rather than IP address. SOCKS5 (Recommended)\nThe recommended option due to its flexibility and features. Supports both IPv4 and IPv6, includes built-in authentication, and can handle various traffic types. Note that some SOCKS5 implementations support UDP traffic but proxychains doesn\u0026rsquo;t support UDP. HTTP/HTTPS\nHTTP proxies were originally designed for web traffic but can handle other TCP connections through the HTTP CONNECT method. The connection to the proxy is encrypted, but once the connect tunnel is established, the proxy sees your traffic in whatever form you send it. If you\u0026rsquo;re accessing HTTP sites, that traffic is still unencrypted inside the tunnel. Protocol Limitations Proxychains intercepts TCP connection functions in dynamically linked programs only. Protocols that don\u0026rsquo;t use these cannot be intercepted.\nUDP Traffic: Generally not supported because UDP is connectionless and bypasses the connect() calls that proxychains intercepts. ICMP Packets: Tools like ping and traceroute can\u0026rsquo;t be proxied since they use ICMP. Raw Sockets: Applications using raw socket operations communicate directly with the kernel, bypassing the libc functions that proxychains intercepts. Chain Modes These are the types of chains / routing that will be used by proxychains.\nDynamic Chain Mode (Recommended) Routes traffic through all listed proxies in sequence. Automatically skips unresponsive proxies.\nRequires at least 1 proxy to be online. Automatically continues operation even when some proxies are unreachable. This is the recommended chain. Commonly used when proxy availability is inconsistent and uptime is critical. Strict Chain Mode Routes traffic through all listed proxies in the exact order specified, otherwise it fails.\nAll proxies must be online. The whole connection fails if any single proxy in the chain is unreachable. This chain ensures a consistent and predictable network path for all connections. Round-Robin Chain Mode Distributes traffic sequentially across all proxies in round-robin rotation.\nRequires at least 1 proxy to be online. Each new connection uses the next proxy in the sequence and circles back to the first proxy after reaching the end of the chain. This chain provides load distribution across the proxies, preventing an individual proxy from being overloaded or having your connection rate-limited. Random Chain Mode Routes traffic through a randomly selected proxy from the configured listed.\nRequires at least 1 proxy to be online. Control how many proxies from the list are used per connection with the chain_len config parameter. This chain enhances anonymity due to the random proxy selection but may have variable performance due to different proxy server location and speed. Configuration File Proxychains looks for its config file in this order:\n$PROXYCHAINS_CONF_FILE environment variable Environment variable for a custom config file path. Takes highest priority when set. -f \u0026lt;config_file_path\u0026gt; command line option Command option for proxychains binary execution. Specifies the config file directly when running proxychains. ~/.proxychains/proxychains.conf or ~/.proxychains/proxychains4.conf User-specific config file path. Filename depends on Linux distro. Overrides system-wide configuration. /etc/proxychains.conf or /etc/proxychains4.conf System-wide config file path. Filename depends on Linux distro. Used when no user-specific config exists. Proxy Lists Free Proxy Services Same as VPNs, I don\u0026rsquo;t recommend free public proxies as they often have security/privacy risks and reliability issues. Instead, you can self-host your own proxy infrastructure (future post) or use a paid one.\nProxyScrape: https://proxyscrape.com/free-proxy-list Proxifly (GitHub): https://github.com/proxifly/free-proxy-list ProxyDB: https://proxydb.net/ GeoNode: https://geonode.com/free-proxy-list Paid Proxy Services Oxylabs: https://oxylabs.io/ Bright Data: https://brightdata.com/ IPBurger: https://www.ipburger.com/ Decodo: https://decodo.com/ Note: These are common proxy services, but do your own research before choosing one.\nGood to Know Information Advantages and Limitations Advantages\nApplication Transparency: Applications work without any configuration changes (they\u0026rsquo;re unaware of the proxy). Protocol Flexibility: You can mix SOCKS4/5 and HTTP proxies in the same chain. Multi-Proxy Chaining: Route traffic through multiple proxies for better anonymity. DNS Privacy: Routes DNS queries through proxies to prevent leaks. Limitations\nProtocol Restrictions: TCP only (limited UDP support). Dynamic Linking Only: Doesn\u0026rsquo;t work with statically compiled binaries. Compatibility Issues: Static binaries bypass the interception mechanism. Apps using dlopen() or raw sockets may not work. Direct syscalls can\u0026rsquo;t be intercepted. AppArmor/SELinux may block functionality. Performance Impact: Added latency per proxy hop. Bandwidth limited by slowest proxy. Use Cases Security Testing\nAnonymize penetration testing tools and vulnerability scans. Pivot through compromised systems to access isolated networks. Perform OSINT gathering and web app testing from different GEOs. Privacy and Operations\nBypass geographic restrictions and censorship. Access corporate resources through proxy tunnels. Maintain operational security during sensitive research. Development \u0026amp; Testing\nTest APIs through different network paths. Simulate distributed user bases across regions. Validate application behavior under various network conditions. Data Collection\nBypass IP-based rate limiting by rotating through multiple proxy endpoints. Gather competitive intelligence anonymously. Avoid region-specific price discrimination. Integration with Tor Proxychains and Tor are often mentioned together, but they\u0026rsquo;re different tools for different purposes. Proxychains is tool that routes your application traffic through proxy servers, while Tor is a dedicated anonymity network with built-in cryptographic protections.\nIf you want to access the Tor network anonymously, just use Tor Browser or torsocks. These tools are designed for anonymous browsing through the Tor network. And use proxychains when you need to route regular application TCP traffic through proxy servers (not on the Tor network).\nWhat NOT to Do:\nDon\u0026rsquo;t Use Tor Browser with Proxychains: Tor Browser already routes traffic through Tor, adding proxychains is redundant and could create potential security risks. Don\u0026rsquo;t Chain Tor with Additional Proxies: Unlike popular believe, adding proxies before or after Tor typically reduces anonymity rather than enhancing it. Don\u0026rsquo;t Run as Root: Running GUI applications as root usually creates security risks. Correct Proxychains Usage with Tor: If you need to route command-line tools through Tor (not for anonymity, but for functionality), configure proxychains to use the local Tor SOCKS proxy:\n# Add the Tor proxy to proxychains.conf socks5 127.0.0.1 9050 # Start Tor service first sudo systemctl start tor # Use proxychains with apps proxychains curl https://check.torproject.org proxychains firefox # For general browsing, NOT FOR TOR BROWSINGThere will be another post dedicated to Tor, but take into consideration some pros and cons of using the Tor network with proxychains.\nWhen you use Tor, your traffic enters through an entry node and eventually exits through an exit node that could be run by anyone (regular people or malicious actors). Yes, you\u0026rsquo;re truly anonymous (the node operator has no way to see your real IP address), but they can see any unencrypted traffic passing through their node. If using using a paid proxy service, you know who the owner of the proxy is and its privacy policy, which creates some peace of mind for some users, but the owner can see your real IP and that you are using Tor. The choice depends on you, if you need anonymity from the destination server and don\u0026rsquo;t want anyone correlating your real identity with your traffic, Tor is the way to go. If you prefer working with accountable infrastructure where you know who\u0026rsquo;s running the servers, a reputable paid proxy service makes more sense. My recommendation is to use a zero-knowledge VPN (like Proton or Mullvad) alongside Tor, which hides your Tor usage from your ISP and prevents Tor entry guards from seeing your real IP address. Proper Anonymity Techniques Use Tor for Anonymity, Not Proxychains: Tor provides cryptographic anonymity. Proxychains is for pivoting and bypassing restrictions, not anonymity. I don\u0026rsquo;t recommend chaining Tor with additional proxies, defeats the purpose. VPN/Proxy Combinations: Correct: VPN -\u0026gt; Tor (VPN then Tor browser or torsocks). Incorrect: VPN -\u0026gt; Proxychains -\u0026gt; Tor (reduces anonymity). Use Trusted Infrastructure: Deploy your own proxy servers on trusted cloud providers. Use reputable commercial proxy services (not free ones). Audit and monitor your proxy infrastructure regularly (check leaks). OpSec Do\u0026rsquo;s:\nVerify if you have any DNS leak. Use end-to-end encryption and HTTPS as much as possible. Rotate proxies regularly. Use application-specific isolation (separate browser profiles, VMs). Don\u0026rsquo;ts:\nDon\u0026rsquo;t reuse the same chain long-term. Don\u0026rsquo;t mix personal and ops/research traffic. Don\u0026rsquo;t assume proxychains provides encryption (it doesn\u0026rsquo;t). Don\u0026rsquo;t run GUI apps as root with proxychains. Working with Proxychains Install Proxychains There are 2 versions of proxychains: the legacy proxychains which is unmaintained, and the modern proxychains-ng (use this one). The naming differs between Linux distros, which can be confusing.\nRHEL-based distros (Fedora, CentOS, etc.): Only proxychains-ng is available. Install with package name proxychains-ng, run with proxychains command, and configure via proxychains.conf. Debian-based distros (Ubuntu, Debian, etc.): Both versions are available. The modern proxychains-ng is packaged as proxychains4 to avoid conflicts with the legacy version. Run with proxychains4 command and configure via proxychains4.conf. Recommendation: Always use proxychains-ng. On Debian systems, make sure you\u0026rsquo;re running proxychains4 (not proxychains - check you are running version 4.x, not 3.x). # RHEL/Fedora dnf install proxychains-ng # Debian/Ubuntu apt install proxychains4Configure Proxychains Edit /etc/proxychains.conf or ~/.proxychains/proxychains.conf.\nvim ~/.proxychains/proxychains.conf# Sample Proxychains Configuration File # ============================================ # Chain Mode Selection (choose ONE) # ============================================ dynamic_chain # Recommended: skips dead proxies # strict_chain # All proxies must be online # round_robin_chain # Rotate through proxies in round-robin # random_chain # Random proxy selection # ============================================ # Random Chain Settings # ============================================ # Only applies when random_chain is enabled chain_len = 2 # Number of proxies per chain # ============================================ # DNS Configuration # ============================================ # Prevent DNS leaks by routing DNS through proxy proxy_dns # Remote DNS subnet for fake local responses # Used internally to identify DNS requests remote_dns_subnet 224 # ============================================ # Connection Timeouts (milliseconds) # ============================================ tcp_read_time_out 15000 # Read timeout: 15 seconds tcp_connect_time_out 8000 # Connect timeout: 8 seconds # ============================================ # Output Control # ============================================ # Suppress proxychains debug output quiet_mode # ============================================ # Local Network Exclusions # ============================================ # Bypass proxy for local/private networks localnet 127.0.0.0/255.0.0.0 # Localhost localnet 10.0.0.0/255.0.0.0 # Private Class A localnet 172.16.0.0/255.240.0.0 # Private Class B localnet 192.168.0.0/255.255.0.0 # Private Class C # ============================================ # Proxy List # ============================================ # In this section is where you list the proxy servers to use [ProxyList] # Format: \u0026lt;type\u0026gt; \u0026lt;host\u0026gt; \u0026lt;port\u0026gt; [username] [password] # SOCKS Proxies (no authentication) socks4 192.168.1.100 1080 socks5 192.168.1.101 1080 # SOCKS5 with authentication socks5 proxy.example.com 1080 username password # Tor SOCKS Proxy (default) socks5 127.0.0.1 9050 # HTTP Proxies http 192.168.1.102 8080 http proxy.example.com 8080 username password # HTTPS Proxy http proxy.example.com 3128 user pass123Using Proxychains Run proxychains alongside another program/app to cycle through proxies. -f \u0026lt;config_file_path\u0026gt;: Specify a custom proxychains.conf file to use. -q: Enable quiet mode (suppress proxychain output). # Syntax proxychains [options] \u0026lt;program\u0026gt; [program_options] # Examples # Simple command execution proxychains nslookup example.com # Launch Firefox proxychains firefox # Network scanning (requires specific flags) proxychains nmap -sT -Pn -n target.com # SSH connection proxychains ssh user@remote-hostTest Proxy Anonymity # Check anonymity level with proxy checker curl -x socks5://proxy.example.com:1080 https://ip-api.com/json/ # Verify no IP leaks proxychains curl https://ipleak.net/json/","permalink":"https://danoss.me/en/categories/security/proxychains-deep-dive/","summary":"Deep dive information on proxychains, use cases, types of chains, configuration, and how to use it securely","title":"Proxychains Deep Dive"},{"content":"\nPart 3 - Using YubiKeys with GPGs.\nNote: All the below configuration is done in Fedora with GNOME DE (applicable 1-to-1 on related distros) and YubiKey 5C / USB (not the Biometric version).\nWhat is GPG? According to the official website:\nGnuPG is a complete and free implementation of the OpenPGP standard as defined by RFC 4880. GnuPG allows you to encrypt and sign your data and communications; it features a versatile key management system, along with access modules for all kinds of public key directories. GnuPG, also known as GPG, is a command line tool with features for easy integration with other applications. A wealth of frontend applications and libraries are available. GnuPG also provides support for S/MIME and Secure Shell (ssh).\nYubiKey GPG Keys When working with GPG keys, YubiKey uses its OpenPGP application to store GPG private keys in its secure element, which prevents them from being extracted under any circumstance.\nSecure Element: Tamper-resistant chip inside YubiKey that stores cryptographic material. Data stored here cannot be extracted, even with physical access to the hardware.\nAll cryptographic operations (like signing, decrypting, authenticating) execute within this secure element and expose only the results.\n# Show Yubikey information, such as enabled applications ykman info# Command output: Device type: YubiKey 5C ... Applications ... FIDO2 Enabled OpenPGP Enabled # This is the application usedKey Slots The OpenPGP YubiKey application has 3 dedicated slots, each serving a specific function.\nSignature Slot: Used for creating digital signatures Encryption Slot: Used for decrypting data Authentication Slot: Used for authentication operations (e.g., SSH) Each slot can hold one RSA, ECC, or Ed25519/Cv25519 key pair.\nAdding GPG Keys to YubiKey The YubiKey OpenPGP application supports 2 ways for provisioning GPG keys:\nOn-Device GPG Key Generation\nKeys are generated in the secure element and never exist outside the YubiKey. Because of this, they are very secure but cannot be backed up. If the YubiKey is lost or damaged, the keys are permanently lost. Generate GPG Key Externally and Then Import To YubiKey\nKeys are generated on your computer and then imported to the YubiKey. This allows you to backup the key, import the same key to multiple YubiKeys, and maintain a master key for subkey rotation (more on that later). Recommended approach for most users. Note: Once imported to the YubiKey, private keys cannot be extracted or exported from the secure element. The device will perform all cryptographic operations internally, exposing only the results. Why Use GPG with YubiKey? Why not? As many things, YubiKey improves security at many levels.\nEven if your computer is compromised, private keys will remain inaccessible to the attacker. Signing/decryption operations require access to the YubiKey.\nThe OpenPGP Key Topology OpenPGP uses a hierarchical trust model where a single Master Key can create specialized Subkeys.\nThis architecture allows SubKeys to be used, stored in the YubiKey, and rotated without losing your cryptographic identity (stored in the Master Key, which should be kept in an offline secured location).\nGPG Key Type Capability Storage Location Description Master Key Certify Secure Offline Storage This is your identity.Used to: create and revoke subkeys, sign other people\u0026rsquo;s keys (web of trust), update key expiration dates.Never store it in the YubiKey.Keep it offline on a secure location, so that if your YubiKey is stolen or damaged, you can use the Master Key to revoke the old subkeys from the YubiKey, generate new subkeys and sign them, maintaining your identity and web of trust. Signing Subkey Sign YubiKey Slot Used to sign data (e.g., Git commits, emails).Requires YubiKey touch and PIN for every signing operation. Encryption Subkey Encrypt YubiKey Slot Used to decrypt files and messages that were encrypted by your public key.Requires YubiKey touch and PIN for every decryption operation. Authentication Subkey Authenticate YubiKey Slot Used for SSH authentication via the GPG Agent and PAM authentication.Some people prefer this over FIDO2 for managing SSH access. Configuration Install Required Packages What Does Each Each Package Do?\ngnupg2: GPG implementation. pinentry-gnome3: GNOME-integrated PIN entry dialog. pcsc-lite: PC/SC smart card middleware. pcsc-lite-ccid: Generic USB CCID smart card reader driver. yubikey-manager: Official Yubico CLI tool (ykman) to manage the YubiKey. # Install GPG and smart card tools sudo dnf install gnupg2 pinentry-gnome3 pcsc-lite pcsc-lite-ccid # Install YubiKey management tools (if not already installed) sudo dnf install yubikey-manager # Enable and start the PC/SC Smart Card daemon sudo systemctl enable --now pcscdGenerate GPG Keys (External Generation) Best practice is to generate GPG keys externally (not on the YubiKey) to allow for backups and then move the Subkeys to the YubiKey.\nIf you already have your GPG keys generated, skip this section.\nGenerate Master Keys My recommendation is to generate Ed25519/Cv25519 keys. # Generate master key and subkeys gpg --expert --full-generate-key # Pretty much all default options are the most secure onesAdd Subkeys # Ger your GPG master key ID with gpg --list-keys # or just put the email specified when generating the master key # Edit the master key to add subkeys # Syntax: gpg --expert --edit-key KEY_ID_OR_EMAIL gpg --expert --edit-key email@example.com # At the gpg\u0026gt; prompt, add 3 subkeys: # 1. Signing subkey [S] gpg\u0026gt; addkey # Choose: (10) ECC (sign only) # Choose: (1) Curve 25519 *default* # Expiration: 1y # 2. Encryption subkey [E] gpg\u0026gt; addkey # Choose: (12) ECC (encrypt only) # Chosse: (1) Curve 25519 *default* # Expiration: 1y # 3. Authentication subkey [A] gpg\u0026gt; addkey # Choose: (11) ECC (set your own capabilities) # Choose: S (toggle off sign capability) # Choose: A (toggle on authenticate capability) # Choose: Q (finish) # Chosse: (1) Curve 25519 *default* # Expiration: 1y # Save and exit - if not saved, changes revert gpg\u0026gt; saveList Generated Keys Show the generated GPG master and subkeys. When listing these keys, you\u0026rsquo;ll see single-letter capability flags, which makes them easy to recognize their function:\n[C] - Certification (Master Key). [S] - Signing. [E] - Encryption. [A] - Authentication. # Show GPG keys generated (master and subkeys) gpg --list-secret-keys --keyid-format LONG email@example.com # Command output # sec ed25519/XXXXXXXX 2025-11-15 [C] # created: 2025-11-15 expires: never usage: C # trust: ultimate validity: ultimate # ssb ed25519/XXXXXXXX 2025-11-15 [S] [expires: 2026-11-15] # ssb cv25519/XXXXXXXX 2025-11-15 [E] [expires: 2026-11-15] # ssb ed25519/XXXXXXXX 2025-11-15 [A] [expires: 2026-11-15]Backup Your GPG Keys (IMPORTANT) Remember that once the subkeys are added to the YubiKey, they cannot be extracted from the YubiKey AND private subkeys are also removed from the local machine keyring, so always backup before transferring.\n# Export private keys (do this BEFORE moving to YubiKey) gpg --export-secret-keys --armor KEY_ID_OR_EMAIL \u0026gt; gpg_master_key.asc gpg --export-secret-subkeys --armor KEY_ID_OR_EMAIL \u0026gt; gpg_subkeys.asc # Export public key gpg --export --armor KEY_ID_OR_EMAIL \u0026gt; gpg_public_key.asc # Export trust database gpg --export-ownertrust \u0026gt; gpg_trust.txt # Store these files on a secure encrypted backup (preferrably offline)Working With YubiKey Check YubiKey OpenPGP Status Before making any changes, verify the current state of your YubiKey\u0026rsquo;s OpenPGP application:\n# View OpenPGP application info ykman openpgp info # Command output: # OpenPGP version: 3.4 # Application version: 5.7.2 # PIN tries remaining: 3 # Reset code tries: 0 # Admin PIN tries: 3 # Require PIN for signature: Once# View detailed card status using GPG gpg --card-status # Command output: # Reader ...........: Yubico YubiKey OTP FIDO CCID 00 00 # Application ID ...: XXXXXXXXXXXXXXXXXXXXXXXXXXXX # Application type .: OpenPGP # Version ..........: 3.4 # Manufacturer .....: Yubico # Serial number ....: 12345678 # Name of cardholder: [not set] # ...Note: If the YubiKey smart card cannot be read, check the Troubleshooting Issues section at the end.\nChange Default PINs (IMPORTANT) The YubiKey ships with factory default PINs that must be changed.\nOpenPGP Application PINs: These are different from the FIDO2 application PINs. User PIN lockout: 3 failed attempts locks the User PIN (can only be unlocked with Admin PIN) Admin PIN lockout: 3 failed attempts permanently locks the OpenPGP application (requires factory reset) PIN Type Purpose Default Value Security Considerations User PIN Daily operations (sign, decrypt, authenticate) 123456 You\u0026rsquo;ll type this frequently so balance security with usability. Admin PIN Admin tasks (key import, settings changes) 12345678 Should be complex and stored in a secure location. # Change User PIN ykman openpgp access change-pin # Enter PIN: (if first time, type the default value: 123456) # New PIN: # Repeat for confirmation: # Change Admin PIN ykman openpgp access change-admin-pin # Enter PIN: (if first time, type the default value: 12345678) # New PIN: # Repeat for confirmation: Move Subkeys to YubiKey The keytocard operation permanently moves your private GPG subkey to the YubiKey and removes it from your GPG keyring. The master key remains on your computer for future subkey management.\nBefore Moving your Subkeys to YubiKey Make Sure That:\nMaster and Subkeys are backed up. Default PINs are changed to strong values. # Edit the key gpg --edit-key KEY_ID_OR_EMAIL # List keys to see subkey numbers gpg\u0026gt; list # Move signing subkey gpg\u0026gt; key 1 # Select 1st subkey (signing) - selection shows ssb* gpg\u0026gt; keytocard # Choose slot: 1 (Signature key) # Enter passphrase and Admin PIN when prompted gpg\u0026gt; key 1 # Deselect key 1 gpg\u0026gt; key 2 # Select 2nd subkey (encryption) - selection shows ssb* gpg\u0026gt; keytocard # Choose slot: 2 (Encryption key) # Enter passphrase and Admin PIN when prompted gpg\u0026gt; key 2 # Deselect key 2 gpg\u0026gt; key 3 # Select 3rd subkey (authentication) - selection shows ssb* gpg\u0026gt; keytocard # Choose slot: 3 (Authentication key) # Enter passphrase and Admin PIN when prompted gpg\u0026gt; key 3 # Deselect key 3 # Save and exit gpg\u0026gt; saveVerify Keys Are on YubiKey Check Card Status\nOutput should show the GPG subkeys. # Check card status gpg --card-status # You should see your 3 subkeys listed: # Signature key ....: [fingerprint] # Encryption key....: [fingerprint] # Authentication key: [fingerprint] # sec ed25519/XXXXXXXX 2025-11-15 [C] # created: 2025-11-15 expires: never usage: C # trust: ultimate validity: ultimate # ssb\u0026gt; ed25519/XXXXXXXX 2025-11-15 [S] [expires: 2026-11-15] # ssb\u0026gt; cv25519/XXXXXXXX 2025-11-15 [E] [expires: 2026-11-15] # ssb\u0026gt; ed25519/XXXXXXXX 2025-11-15 [A] [expires: 2026-11-15]List Secret Keys\nIn the output, notice the \u0026gt; symbol after ssb - this confirms the subkeys are now stored on the YubiKey, not in your GPG keyring. # List secret keys (should show \u0026#34;ssb\u0026gt;\u0026#34; indicating keys are on card) gpg --list-secret-keys # Command output # sec ed25519/XXXXXXXX 2025-11-15 [C] # created: 2025-11-15 expires: never usage: C # trust: ultimate validity: ultimate # ssb\u0026gt; ed25519/XXXXXXXX 2025-11-15 [S] [expires: 2026-11-15] # ssb\u0026gt; cv25519/XXXXXXXX 2025-11-15 [E] [expires: 2026-11-15] # ssb\u0026gt; ed25519/XXXXXXXX 2025-11-15 [A] [expires: 2026-11-15]Configure Touch Requirement (IMPORTANT) Requiring a physical touch for each GPG operation prevents malware from using your keys without explicit physical contact (even if it captures your PIN).\n# Require touch for all operations ykman openpgp keys set-touch sig on # Signing ykman openpgp keys set-touch enc on # Encryption ykman openpgp keys set-touch aut on # Authentication # Options: # on - Touch required, but cached for 15 seconds # off - No touch required # fixed - Touch required every time, no caching (most secure) # cached - Touch required, cached until YubiKey is removedPIN Management (OPTIONAL) Set retry limits. ykman openpgp access set-retries 5 5 5Remove Master Key From Local Keyring (OPTIONAL) Export Private Master Key\nExport the private master key and back it up in a secure offline location. For maximum security, this master key should be kept offline, not on your computer. You will be working with the subkeys. gpg --export-secret-keys -a email@example.com \u0026gt; gpg_master_key.asc # Then save it to an encrypted USB drive or something secureRemove Master Key from Computer\n# Delete master secret key from GPG keyring # Ensure you have offline backups before doing this!! gpg --delete-secret-keys email@example.com # Re-import only your public keys and card stubs (from previous sections) gpg --import gpg_public_key.ascVerify Master Key Is Removed\nThe # symbol next to the private key means the master key is not available. # Verify: master key should show \u0026#39;sec#\u0026#39; gpg --list-secret-keys # Command output # sec# ed25519/XXXXXXXX 2025-11-15 [C] # The master key is offline # created: 2025-11-15 expires: never usage: C # trust: ultimate validity: ultimate # ssb\u0026gt; ed25519/XXXXXXXX 2025-11-15 [S] [expires: 2026-11-15] # ssb\u0026gt; cv25519/XXXXXXXX 2025-11-15 [E] [expires: 2026-11-15] # ssb\u0026gt; ed25519/XXXXXXXX 2025-11-15 [A] [expires: 2026-11-15]Configure GPG Agent Configure the GPG agent to manage PIN caching, SSH support, and PIN entry dialogs.\nEdit ~/.gnupg/gpg-agent.conf\n# Edit ~/.gnupg/gpg-agent.conf vim ~/.gnupg/gpg-agent.conf# =================================================================== # PIN Entry Program # =================================================================== pinentry-program /usr/bin/pinentry-gnome3 # Alternatives: # pinentry-program /usr/bin/pinentry # Original vlue # pinentry-program /usr/bin/pinentry-curses # Terminal-only # pinentry-program /usr/bin/pinentry-qt # KDE/Qt environments # =================================================================== # SSH Support # =================================================================== enable-ssh-support # =================================================================== # GPG Key Passphrase Caching (disk-based keys) # =================================================================== default-cache-ttl 3600 # 1 hour idle timeout max-cache-ttl 7200 # 2 hour absolute maximum # =================================================================== # SSH Key Passphrase Caching # =================================================================== default-cache-ttl-ssh 3600 # 1 hour idle timeout max-cache-ttl-ssh 7200 # 2 hour absolute maximum # =================================================================== # Security Settings # =================================================================== no-allow-external-cache no-allow-mark-trusted # =================================================================== # Display Settings # =================================================================== keep-display keep-ttySet Secure Permissions\nchmod 600 ~/.gnupg/gpg-agent.confRestart the GPG Agent To Implement Changes\n# Kill existing agent gpgconf --kill gpg-agent # Start new agent (happens automatically on next GPG operation) gpg-agent --daemon # Verify agent is running gpgconf --list-dirs agent-socketEnable SSH Support (Optional) If you plan to use the Authentication subkey for SSH login, point the SSH_AUTH_SOCK environment variable to point to the GPG agent.\nCreate Variable Definition\nAdd the variable export to as script in ~/.bashrc.d/ to make it persistent. Make sure your ~/.bashrc file reads this drop-in directory - which most distros do. # Ensure regular user\u0026#39;s .bashrc sources ~/.bashrc.d (optional) grep -q \u0026#34;bashrc.d\u0026#34; ~/.bashrc || echo \u0026#39; # Source custom scripts from .bashrc.d/ if [ -d ~/.bashrc.d ]; then for i in ~/.bashrc.d/*.sh; do [ -r \u0026#34;$i\u0026#34; ] \u0026amp;\u0026amp; source \u0026#34;$i\u0026#34; done fi\u0026#39; \u0026gt;\u0026gt; ~/.bashrc # Create bashrc.d/ drop-in directory (if not exist) mkdir -p ~/.bashrc.d/ # Create script with export definition in drop-in directory cat \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; \u0026gt; ~/.bashrc.d/variables.sh #!/bin/bash # Use GPG Agent as SSH agent export SSH_AUTH_SOCK=$(gpgconf --list-dirs agent-ssh-socket) EOF # Reload shell configuration to implement changes source ~/.bashrcVerify SSH Support is Active\n# Verify SSH support is active ssh-add -L # Should display: ssh-ed25519 XXXX... cardno:0000000000Configure SSH to Use GPG Key\nNow, just export the SSH public key from your GPG private key and add it to the remote target SSH server\u0026rsquo;s authorized_keys file to connect. # Export the SSH public key from GPG gpg --export-ssh-key your.email@example.com \u0026gt; ~/.ssh/yubikey.pub # Add the SSH public key to remote servers\u0026#39; authorized_keys cat ~/.ssh/yubikey.pub | ssh user@server \u0026#39;cat \u0026gt;\u0026gt; ~/.ssh/authorized_keys\u0026#39;Connect Using SSH\nWhen connecting, you will be prompted to enter the User PIN and to touch the YubiKey metal plate immediately to login. If you only enter the PIN and don\u0026rsquo;t touch it you will get an error similar to: sign_and_send_pubkey: signing failed for ED25519 \u0026quot;cardno:32_740_587\u0026quot; from agent: agent refused operation. # Login as usual via SSH ssh user@serverUsing YubiKey on New Systems When you use a new computer or need to restore access, you only need your public key and the YubiKey.\nInstall Required Packages # Install GPG and smart card tools sudo dnf install gnupg2 pinentry-gnome3 pcsc-lite pcsc-lite-ccid # Install YubiKey management tools (if not already installed) sudo dnf install yubikey-manager # Enable and start the PC/SC Smart Card daemon sudo systemctl enable --now pcscdImport Public Key First import your GPG public key from the backup or from a keyserver (if you uploaded it - I recommend https://keys.openpgp.org/). # Import your public key from a file gpg --import gpg_public_key.asc # Or get from keyserver (if you uploaded it) gpg --keyserver hkps://keys.openpgp.org --recv-keys KEY_ID_OR_EMAIL # Trust the key (set to ultimate trust because its your own key) gpg --edit-key KEY_ID_OR_EMAIL gpg\u0026gt; trust # Choose: 5 = I trust ultimately gpg\u0026gt; quitTest YubiKey GPG Operations Verify YubiKey is Detected\nIf it fails, check the troubleshooting section. # Verify YubiKey is detected gpg --card-statusTest Signing Operation\n# Test signing echo \u0026#34;test\u0026#34; | gpg --clearsign # You should be prompted for: # 1. User PIN # 2. Touch on YubiKey (if enabled)Test Encryption and Decryption Operations\n# Test encryption and decryption (to yourself) echo \u0026#34;secret\u0026#34; | gpg --encrypt -a --recipient email@example.com \u0026gt; test.asc # Decrypt the new encrypted file gpg --decrypt test.asc # Remove the test file rm test.ascTest SSH (if configured)\n# List SSH keys managed by GPG agent ssh-add -L # Expected output: # ssh-ed25519 XXXXXXXXXXXXXXXXXXXXXXXX... cardno:000000 # Test SSH connection to a server (remote server needs the public key) ssh -v user@remote-server # You should be prompted for: # 1. User PIN (first use only, then cached) # 2. Touch on YubiKey (if enabled)Troubleshooting Issues During my first setup, I encountered an issue where the GPG card was not found and the gpg --card-status command returned the error below.\ngpg --card-status # Command output: # gpg: selecting card failed: No such device # gpg: OpenPGP card not available: No such deviceFirst, check the status of the pcscd service. If it shows a permission issue, you have to update polkit to allow the current user access.\n# Check the stauts of pcscd service systemctl status pcscd # Command output: # ... systemd[1]: Started pcscd.service - PC/SC Smart Card Daemon. # ... pcscd[2224]: 00000000 ../src/auth.c:166:IsClientAuthorized() Process 2162 (user: 60578) is NOT authorized for\u0026gt; # ... pcscd[2224]: 00000224 ../src/winscard_svc.c:357:ContextThread() Rejected unauthorized PC/SC client # ... pcscd[2224]: 00022773 ../src/auth.c:166:IsClientAuthorized() Process 2162 (user: 60578) is NOT authorized for\u0026gt; # ... pcscd[2224]: 00000384 ../src/winscard_svc.c:357:ContextThread() Rejected unauthorized PC/SC client # Create a polkit rule to allow your user access to pcscd sudo vim /etc/polkit-1/rules.d/99-pcscd.rules # Add the below to the 99-pcscd.rules file polkit.addRule(function(action, subject) { if (action.id == \u0026#34;org.debian.pcsc-lite.access_pcsc\u0026#34; || action.id == \u0026#34;org.debian.pcsc-lite.access_card\u0026#34;) { return polkit.Result.YES; } }); # Set more secure permissions to the file sudo chmod 644 /etc/polkit-1/rules.d/99-pcscd.rules # Restart polkit and pcscd to implement the changes sudo systemctl restart polkit.service sudo systemctl restart pcscd.service # Test again (should work now) gpg --card-statusBy default, the pcscd.service is activated via a systemd socket. Make sure to start the socket automatically at boot.\n# Enable and start the pcscd socket sudo systemctl enable --now pcscd.socketWhat\u0026rsquo;s Next? YubiKeys are really useful and really simple to setup. The only concern is losing the key, so my recommendation is to always have a backup in a secure location and use PINs for every operation.\nDeep dives on GPG keys might come in the future but it\u0026rsquo;s pretty simple as well. Next topics might be proxychains, Docker/Podman container-related topics, or self-hosting.\n","permalink":"https://danoss.me/en/categories/security/yubikeys-part-3-gpg/","summary":"How to configure and use Yubikey with GPG","title":"Yubikeys Part. 3 - GPG"},{"content":"\nPart 2 - Using YubiKey with SSH to authenticate to remote SSH servers. Not gonna explain what SSH is and use cases or configuration, but might do a deep-dive if someone is interested.\nNote: All configuration is done in Fedora with GNOME DE (applicable 1-to-1 on related distros) and YubiKey 5C / USB (not the Biometric version).\nSSH Authentication with YubiKey FIDO2 What is FIDO2? FIDO2 is an open authentication standard that lets you use hardware security keys (like YubiKey) for passwordless authentication.\nOpenSSH + FIDO2\nOpenSSH added support for FIDO2 on version 8.2, meaning we can now use the YubiKey alongside (or instead of) traditional password or key-based authentication.\nWhen you authenticate, the cryptographic operations happen directly on the hardware token itself. So even if someone compromises your computer and steals your SSH key files, they still can\u0026rsquo;t authenticate without physically having your YubiKey.\nThis offers another layer of security. Every single authentication requires you to physically touch your YubiKey (you can also configure it to require PIN verification if you want more security).\nFIDO2 SSH Key Types There are 2 types of FIDO2 SSH keys:\nNon-Resident Keys:\nThe private key is stored as a file on the computer alongside your other SSH keys. This is just like a traditional SSH key, however, the key is useless by itself. It requires your YubiKey to perform any operation (like signing in). These keys don\u0026rsquo;t consume any of your YubiKey\u0026rsquo;s credential slots (you can create as many as you want). To authenticate on a SSH server, the private key must be present in your computer and the public key must be present in the target SSH server\u0026rsquo;s authorized_keys file. Resident Keys:\nThe private key is stored directly in the YubiKey\u0026rsquo;s internal memory. No key files are stored on the computer. These keys consume one of your YubiKey\u0026rsquo;s credential slots (YubiKey 5 holds up to 100). So, if you create resident keys for connecting to dozens of servers, you can run out of slots. To authenticate on a SSH server, you can just plug your YubiKey into any computer (even new) and the SSH agent can \u0026ldquo;discover\u0026rdquo; the key on the device and use it to authenticate to the target SSH server with the corresponding public key in its authorized_keys file. Configuration Requirements OpenSSH 8.2 or newer for FIDO2 support. The yubikey-manager package, which provides the ykman CLI tool to manage the YubiKey FIDO2 application and more. # Install SSH sudo dnf install openssh-clients # Check OpenSSH version ssh -V # Install the \u0026#34;yubikey-manager\u0026#34; package if not already installed sudo dnf install yubikey-managerSet YubiKey FIDO2 PIN Before generating keys or using PIN verification, you must set a FIDO2 PIN on your YubiKey.\n# Change or set your FIDO2 PIN ykman fido access change-pin # Verify your PIN works ykman fido access verify-pinImportant: This PIN is separate from PIV PINs and OpenPGP PINs. Each YubiKey application has its own PIN.\nGenerate SSH Keys Non-Resident Key (Recommended) Non-resident keys don\u0026rsquo;t consume credential slots on the YubiKey, and if the YubiKey is lost or stolen, an attacker cannot obtain the SSH private key from the YubiKey alone.\nKey Generation Options:\n-t \u0026lt;type\u0026gt;: Type of key algorithm to use. ed25519-sk: Use the Ed25519 algorithm (recommended). ecdsa-sk: Alternative if Ed25519 is not supported, usually for old servers. -O verify-required: Requires both PIN and YubiKey touch for every use, else only touch is needed. -C \u0026quot;comment\u0026quot;: Add a comment to identify the key. This can be any string but the recommendation is to specify something to identify who create the key and where it was generated. # Generate non-resident key with PIN verification ssh-keygen -t ed25519-sk -O verify-required -C \u0026#34;user@hostname\u0026#34; # You\u0026#39;ll be prompted to: # 1. Touch your YubiKey metal plate # 3. Choose a passphrase for the key file (optional but recommended)Resident Key (Portable) Resident keys are stored on the YubiKey itself, enabling portability across machines. They require a PIN to generate and use.\nKey Generation Options:\n-t \u0026lt;type\u0026gt;: Type of key algorithm to use. ed25519-sk: Use the Ed25519 algorithm (recommended). ecdsa-sk: Alternative if Ed25519 is not supported, usually for old servers. -O \u0026lt;option\u0026gt; -O resident: Creates a resident key stored on the YubiKey. -O application=ssh:hostname: Optional label for multiple SSH credentials. Mainly used to differentiate between all keys stored in the YubiKey. -O verify-required: Requires both PIN and YubiKey touch for every use, else only touch is needed. -C \u0026quot;comment\u0026quot;: Add a comment to identify the key. This can be any string but the recommendation is to specify something to identify who created the key and where it was generated. # Generate resident key with PIN verification ssh-keygen -t ed25519-sk -O resident -O verify-required -O application=ssh:hostname -C \u0026#34;user@hostname\u0026#34; # You\u0026#39;ll be prompted to: # 1. Enter your FIDO2 PIN (if verify-required is used) # 2. Touch your YubiKey metal plate # 3. Choose a passphrase for the key file (optional but recommended)Add Public Key to SSH Servers This step is identical for BOTH key types.\nAfter you generate a key (any type), you get a .pub public key file (e.g., id_ed25519_sk.pub), which needs be added to the target SSH server\u0026rsquo;s ~/.ssh/authorized_keys file you want to connect.\n# Copy generated public key to remote server ssh-copy-id -i ~/.ssh/id_ed25519_sk.pub user@remote-server # Or manually append to authorized_keys cat ~/.ssh/id_ed25519_sk.pub | ssh user@remote-server \u0026#34;mkdir -p ~/.ssh \u0026amp;\u0026amp; cat \u0026gt;\u0026gt; ~/.ssh/authorized_keys\u0026#34;Add Keys to SSH Agent Once the public key is added to the target SSH server you want to connect, follow these steps:\nNon-Resident Keys: You need the private key file on your computer to authenticate. Adding the key to the SSH agent is optional. Resident Keys (Agent Required): You need to configure the SSH Agent to fetch the key directly from the YubiKey memory to authenticate. Adding the key to the agent is a requirement for resident keys. This can be done on any new machine you want to connect from. You only need the public key to be in the target SSH server. # Start ssh-agent (if not already running) eval \u0026#34;$(ssh-agent -s)\u0026#34; # Non-Resident: Add private key file to SSH agent ssh-add ~/.ssh/id_ed25519_sk # Resident (on any new machine): Discover and load the key from the YubiKey ssh-add -k # List loaded keys ssh-add -lUsing SSH with YubiKey Once your public key is added to the target SSH server\u0026rsquo;s authorized_keys file, (and loaded to the SSH agent for resident keys), connecting is simple.\n# SSH will automatically detect and use the YubiKey for authentication ssh user@remote-server # Explicitly specify the Yubikey-generated key ssh -i ~/.ssh/id_ed25519_sk user@remote-server # You\u0026#39;ll be prompted to: # 1. Enter your FIDO2 PIN (if verify-required was set) # 2. Touch your YubiKeyImportant Note for GNOME Users\nBy default, the GNOME Linux desktop environment uses its own SSH agent (SSH_AUTH_SOCK=/run/user/1000/gcr/ssh), which doesn\u0026rsquo;t work well with FIDO2 hardware keys. This usually results in a failure to authenticate with the error: sign_and_send_pubkey: signing failed for ED25519-SK \u0026quot;/home/user/.ssh/id_ed25519_sk\u0026quot; from agent: agent refused operation.\nTo fix this, just add the -o \u0026quot;IdentityAgent=none\u0026quot; option when connecting to the target machine. This bypasses the default GNOME agent to authenticate correctly.\n# Connect in a GNOME Linux environment ssh -o \u0026#34;IdentityAgent=none\u0026#34; -i ~/.ssh/id_ed25519_sk user@remote-server # You\u0026#39;ll be prompted to: # 1. Enter your FIDO2 PIN (if verify-required was set) # 2. Touch your YubiKeyThis might look like a very long command just to connect to an SSH server, but you can specify these connection options in your SSH client config file ~/.ssh/config to avoid typing so much for every connection.\nAnd that\u0026rsquo;s it, super simple and now you are more secure!\nDelete Resident Keys From YubiKey General YubiKey management commands are in Part. 1, to make this post more complete, I added them below.\nList Resident Keys\n# List resident keys in YubiKey ykman fido credentials list # Example output Credential ID RP ID Username Display name 5f85afc4... ssh: openssh openssh Delete Resident Keys\n# Delete a resident key from YubiKey by specifying its credential ID # Delete resident key with credential ID \u0026#34;5f85afc4\u0026#34; ykman fido credentials delete 5f85afc4What\u0026rsquo;s Next? Next post will be YubiKeys setup for GPG/PGP/PPP/GGG\u0026hellip;\n","permalink":"https://danoss.me/en/categories/security/yubikeys-part-2-ssh/","summary":"How to configure and use Yubikey for SSH authentication","title":"Yubikeys Part. 2 - SSH"},{"content":"\nRecently bought a pair of YubiKeys (YubiKey 5C specifically), after years of thinking about it in the back of my head\u0026hellip;\nThe initial idea was a single post for my YubiKey setup, but became it too long, so it will be a 2-parter:\nPart 1: YubiKey basics and authentication for local Linux machines. Part 2: Authentication for SSH, and GPG. Note: All configuration is done in Fedora with GNOME DE (applicable 1-to-1 on related distros) and YubiKey 5C / USB (not the Biometric version).\nWhat is a YubiKey? According to Wikipedia, the ultimate source of truth:\nYubiKey is a small hardware device that can \u0026ldquo;protect access to computers, networks, and online services that supports one-time passwords (OTP), public-key cryptography, authentication, and the Universal 2nd Factor (U2F) and FIDO2 protocols developed by the FIDO Alliance. It allows users to securely log into their accounts by emitting one-time passwords or using a FIDO-based public/private key pair generated by the device.\nManaging YubiKeys Install Required Packages The yubikey-manager package provides the ykman CLI tool to manage all YubiKey applications, such as Yubico OTP, FIDO U2F, FIDO2, OATH, PIV, OpenPGP, and YubiHSM Auth.\n# Install the \u0026#34;yubikey-manager\u0026#34; package sudo dnf install yubikey-managerGeneral Commands # Show YubiKey information (firmware, serial, applications) ykman info # List connected YubiKeys ykman list # Enable applications (such as, FIDO2, PIV, etc.) ykman config usb --enable \u0026lt;application_name\u0026gt; # Disable applications (such as, FIDO2, PIV, etc.) ykman config usb --disable \u0026lt;application_name\u0026gt;FIDO Commands Manage FIDO2/U2F authentication (used for local Linux authentication).\n# Show FIDO information (PIN, credential count) ykman fido info # Change or set FIDO2 PIN (required if using PIN verification) ykman fido access change-pin # Verify existing FIDO2 PIN (if already set) ykman fido access verify-pin # Unlock FIDO (after too many failed PIN attempts) ykman fido access unlock # List resident credentials ykman fido credentials list # Delete a credential ykman fido credentials delete # Reset FIDO (WIPES ALL FIDO DATA) ykman fido resetPIV Commands Smart card functionality for certificates, authentication, signing.\n# Show PIV information ykman piv info # Change PIN (default: 123456) ykman piv access change-pin # Change PUK (PIN Unblocking Key, default: 12345678) ykman piv access change-puk # Change management key ykman piv access change-management-key # Generate key in slot ykman piv keys generate 9a /path/to/public.pem # Import certificate ykman piv certificates import 9a /path/to/cert.pem # Export certificate ykman piv certificates export 9a /path/to/output.pem # Generate self-signed certificate ykman piv certificates generate 9a # Reset PIV application (WIPES ALL PIV DATA) ykman piv resetGPG Commands # Show OpenPGP information ykman openpgp info # Change User PIN (default: 123456) ykman openpgp access change-pin # Change Admin PIN (default: 12345678) ykman openpgp access change-admin-pin # Set reset code for PIN recovery ykman openpgp access set-reset-code # Reset OpenPGP application (WIPES ALL PGP DATA) ykman openpgp resetUsing YubiKey for Local Linux Authentication Configure your Linux machine to authenticate/login with just a touch of the YubiKey, no password needed.\nInstall Required Packages pam-u2f provides the PAM module that handles YubiKey authentication. pamu2fcfg is the configuration tool we\u0026rsquo;ll use to register our keys with the system. sudo dnf install pam-u2f pamu2fcfgRegister Your YubiKey(s) Before we can use the YubiKey for local authentication, register it with the system.\nFirst, plug the YubiKey to your machine and run pamu2fcfg. When you run pamu2fcfg, the YubiKey will start blinking. Touch the metal contact to complete the registration.\n# Create the YubiKey configuration directory mkdir -p ~/.config/Yubico # Register your YubiKey (plug it in first and touch the device) pamu2fcfg \u0026gt; ~/.config/Yubico/u2f_keysIf you have a backup YubiKey, register it here too:\n# Add additional backup YubiKeys to the same file (append \u0026gt;\u0026gt;) pamu2fcfg -n \u0026gt;\u0026gt; ~/.config/Yubico/u2f_keysSecure your u2f_keys file to prevent unauthorized access.\nchmod 600 ~/.config/Yubico/u2f_keysImportant Notes:\nThe above key registration is for the current user only. For system-wide YubiKey registration, register the key in the /etc/Yubico/u2f_keys file instead with each user entry per line. Backup the u2f_keys file to a secure location. Configuring PAM Authentication Now its time to configure PAM (Pluggable Authentication Module) to authenticate with the YubiKey. PAM can be scary like a molester in the park, but we\u0026rsquo;ll make it easy.\nPAM controls how authentication works on Linux, so we\u0026rsquo;ll modify a few configuration files to make that happen.\nCommon Configuration Options Authentication Modes\nMode Description required YubiKey auth MUST succeed.If it fails, authentication will ultimately fail (password auth or other remaining modules may still be evaluated).Most secure, but can be a pain if you forget your key. sufficient YubiKey OR password auth will work to authenticate.Slightly less secure. Common Options:\nOption Description interactive Display prompt (Insert your FIDO authenticator, then press ENTER) asking you to insert the key.Displayed before asking you to touch the key to authenticate. cue Display prompt (Please touch the FIDO authenticator) asking you to touch the key to authenticate. pinverification=1 Ask for the YubiKey FIDO2 PIN to authenticate.More secure than touch-only.Requires a FIDO2 PIN to be set (ykman fido access change-pin). nouserok Allows users without registered YubiKeys to bypass YubiKey auth and fall through to other PAM modules (like password auth).Useful in multi-user systems where not everyone has the YubiKey. Configure sudo Access Edit /etc/pam.d/sudo to configure YubiKey authentication when usingsudo.\n# Contents of \u0026#34;/etc/pam.d/sudo\u0026#34; # Add a line like this before any existing auth lines (use \u0026#39;required\u0026#39; or \u0026#39;sufficient\u0026#39; authentication mode and use whichever options you prefer) auth\trequired\tpam_u2f.so cueFor example, a complete /etc/pam.d/sudo file would look like this:\n# Contents of \u0026#34;/etc/pam.d/sudo\u0026#34; #%PAM-1.0 auth\tsufficient\tpam_u2f.so cue pinverification=1 auth\tinclude\tsystem-auth account\tinclude\tsystem-auth password\tinclude\tsystem-auth session\tinclude\tsystem-authImportant:\nMy recommendation is to have the options cue pinverification=1 to authenticate by touching the YubiKey and providing the PIN, which is much more secure than just touching the key. The order of PAM modules is important, keep an eye on it. Test sudo access in a separate terminal before closing your current session to prevent misconfigurations that could lock yourself out. Start with sufficient authentication mode if you don\u0026rsquo;t know what you are doing. Additional Configurations You can apply the same configuration on other authentication points as well:\n/etc/pam.d/gdm-password Authenticate for the graphical login (GDM - GNOME Display Manager). Allows unlocking your desktop with just a YubiKey touch. /etc/pam.d/login Authenticate for console login. /etc/pam.d/polkit-1 Authentication for GNOME privilege elevation prompts (like systemctl). These prompts are controlled by polkit, not the regular PAM authentication. Note on the GNOME Keyring: By default, YubiKey will not unlock the GNOME Keyring. This is because the keyring is encrypted with your user password, and the YubiKey doesn\u0026rsquo;t provide that password to PAM. To override this, you can set a blank password for the keyring, but I personally don\u0026rsquo;t recommend this, just keep the keyring password protected, it\u0026rsquo;s more secure.\nWhat\u0026rsquo;s Next? YubiKeys are easy to setup and offer great security. Just make sure you don\u0026rsquo;t lose them and have a backup key registered in case something happens.\nFor me however, they don\u0026rsquo;t replace 2FA from TOTP applications. They are a compliment due to its ease of use.\nNext post will be YubiKeys setup for SSH and GPG authentication.\n","permalink":"https://danoss.me/en/categories/security/yubikeys-part-1/","summary":"What is Yubikey? How to configure the device, general management, and how to use Yubikey for Linux authentication","title":"Yubikeys Part. 1"},{"content":"\nSomething that often catches people off guard when using a desktop Linux distribution is that Flatpak applications consume a lot of storage space.\nA handful of applications that would take a few hundred megabytes as traditional packages somehow balloon into multiple gigabytes when installed via Flatpak.\nSo what\u0026rsquo;s going on? Why does installing a simple text editor as Flatpak requires hundreds of MB when the native package version requires minimal storage space?\nThe Dependency Problem Flatpak Solves On traditional Linux systems, applications rely on shared system libraries. This is efficient, for example, if 10 applications all need the same library, it\u0026rsquo;s only stored once on disk.\nThis however, creates a couple of problems:\nVersion Conflicts: App A needs library version 1.2, but app B needs version 1.5. Distribution Fragmentation: An app built for Ubuntu might not work on Fedora due to different library versions. System Updates Breaking Apps: A system library update could break apps that relied on the old version. For Flatpaks, each application brings its own dependencies and runs in an isolated sandbox. No more dependency conflicts, and no more worrying about system updates breaking things, however the trade-off is storage space\u0026hellip;\nHow Flatpak Uses Storage Flatpak\u0026rsquo;s storage model consists of 3 main components:\nApplications These are the actual programs you install. Each Flatpak application is self-contained in its own sandbox and includes any dependencies that aren\u0026rsquo;t provided by its runtime.\nRuntimes Runtimes are shared foundations that provide common dependencies for applications. Instead of every single app bundling GTK, Qt, or core system libraries, apps can depend on a runtime that provides these.\nThe major runtimes include:\nFreedesktop: The base runtime providing core libraries and toolchains. GNOME: Built on Freedesktop, adds GNOME-specific libraries. KDE: Built on Freedesktop, adds Qt and KDE libraries. Repository Flatpak uses OSTree to store files in /var/lib/flatpak for system-wide installations or ~/.local/share/flatpak for user installations. This repository contains all the actual data that apps and runtimes are built from.\nWhy the Numbers Look Worse Than They Are Flatpak uses hard links for deduplication. When multiple apps or runtimes share the same file, Flatpak doesn\u0026rsquo;t duplicate it - it creates hard links pointing to the same data on disk. This means a file might appear in multiple places, but it\u0026rsquo;s only stored once.\nThe problem? Standard tools like du count hard links separately, making it appear that you\u0026rsquo;re using far more space than you actually are.\nFor example, if you run:\ndu -sh /var/lib/flatpakYou might see something like 15 GB. But the actual disk usage (counting deduplicated data) could be closer to 8-10 GB. Tools that understand hard links will give you a more accurate picture.\nDeduplication in Flatpaks Flatpak\u0026rsquo;s deduplication works at multiple levels:\nBetween Runtimes: The GNOME and KDE runtimes are both build on the Freedesktop base runtime, so they share those common files rather than duplicating them. Between Applications: Apps using the same runtime share those runtime files. Installing a second GNOME app doesn\u0026rsquo;t require downloading the entire GNOME runtime again. Between versions: When runtimes update, only the changed files are downloaded and stored. Unchanged files are shared between versions. This is why the first Flatpak application installation might be huge (the entire runtime is downloaded), but subsequent installs are much smaller.\nThe Real Storage Cost Despite deduplication, Flatpak does consume more space than traditional package installations because:\nMultiple Runtime Versions: You might have GNOME runtime 48, and 49 all installed because different apps require different versions. Each version can be hundreds of megabytes. Bundled Dependencies: Apps still bundle dependencies that aren\u0026rsquo;t in runtimes. An Electron app, for example, might bundle the entire Electron framework, which can take a lot of space. Leftover Runtimes When you uninstall apps, their runtimes might remain on your system until you manually clean them up. Managing Flatpak Storage Check What\u0026rsquo;s Installed See what\u0026rsquo;s consuming space:\n# List all installed Flatpaks with their sizes flatpak list --columns=name,application,size # List only applications (not runtimes) flatpak list --app --columns=name,size # List only runtimes flatpak list --runtime --columns=name,sizeRemove Unused Runtimes When you uninstall apps, their runtimes might stick around (remove them if not needed):\n# Remove unused runtimes and extensions flatpak uninstall --unused # This is safe and only removes runtimes not used by any installed appMove to a Different Partition If your root partition is filling up, you can install Flatpaks to your home partition instead:\n# Add the user repository flatpak remote-add --user flathub https://flathub.org/repo/flathub.flatpakrepo # Install apps with the --user flag flatpak install --user flathub org.mozilla.firefoxIs the Space Worth It? As the universal answer to many questions, it depends\u0026hellip;\nWhen Flatpak Makes Sense:\nYou want the latest versions of applications without waiting for distribution updates. You need apps that aren\u0026rsquo;t available in your distro\u0026rsquo;s repositories. You value the sandboxing and security isolation of Flatpaks. When Traditional Packages Might be Better:\nYou have limited storage space. You prefer minimal systems with tight control over what\u0026rsquo;s installed. The apps you need are already available and up-to-date in your distro\u0026rsquo;s repos. Final Thoughts I personally use a hybrid approach. Flatpak is my default for most desktop applications, as the sandboxing alone makes it worthwhile for anything that touches the internet or handles untrusted files. However, I\u0026rsquo;m lazy and don\u0026rsquo;t feel the trouble is worthed for some applications that need deeper system integration that struggle with Flatpak isolation/permissions (which can be configured to allow access as needed).\nThe storage overhead is real, however today, with cheap SSD/HDDs, the tradeoff usually makes sense most of the time. This however, depends on your use case (and patience).\n","permalink":"https://danoss.me/en/categories/linux/why-flatpaks-take-so-much-storage-space/","summary":"How Flatpak storage works and why do they take so much space?","title":"Why Flatpaks Take So Much Storage Space"},{"content":"\nI\u0026rsquo;ve been using Bitwarden as my password manager for a couple of years now as its reliable, open-source, easy to use across multiple devices, and secure.\nNow, recently I discovered that Bitwarden added an SSH agent functionality and wanted to try it out. This post walks through setting up and using Bitwarden\u0026rsquo;s SSH agent on Linux.\nWhy Use Bitwarden as an SSH Agent? You might be wondering why use Bitwarden for SSH when traditional SSH key management works fine?\nWell, having your SSH keys in the same secure vault as your passwords means everything is encrypted, backed up, and synced across devices automatically.\nSecond, Bitwarden can prompt you for authorization before allowing an SSH connection, adding an extra layer of security beyond a simple passphrase-protected key file.\nFinally, if you already use Bitwarden across multiple machines, this centralizes your key management into a service that you already use, which is pretty good tbh.\nHow Bitwarden SSH Agent Works Bitwarden SSH agent consists of 3 main components:\nBitwarden Desktop App: Acts as the SSH agent and stores your keys. SSH Agent Socket: Unix domain socket that handles communication between SSH clients and Bitwarden. Remote SSH Servers: Target systems configured for public key authentication. When you attempt an SSH connection, the SSH client communicates with Bitwarden through the socket. Bitwarden can prompt you for authorization (depending on your settings), and provides the private key to complete the authentication.\nSetting Up the Client Install Bitwarden Desktop You can install Bitwarden from a Flatpak or build it from source. I went with the Flatpak version:\n# Install from Flathub flatpak install flathub com.bitwarden.desktopFor other installation methods, check the official documentation.\nEnable the SSH Agent Once installed, you need to enable the SSH agent feature:\nOpen the Bitwarden desktop app Navigate to File → Settings Check the Enable SSH-Agent option Configure the SSH Agent Socket The SSH client needs to know where to find Bitwarden\u0026rsquo;s SSH agent socket. First, locate the socket file:\n# Find the location of the Bitwarden socket find / -name \u0026#34;*bitwarden*\u0026#34; -type s 2\u0026gt;/dev/null # Alternative method using ss ss -xl | grep bitwardenFor a Flatpak installation, the socket is typically located at:\n/home/\u0026lt;username\u0026gt;/.var/app/com.bitwarden.desktop/data/.bitwarden-ssh-agent.sockSet the SSH_AUTH_SOCK environment variable to point to this socket:\n# Syntax export SSH_AUTH_SOCK=\u0026lt;bitwarden_socket_path\u0026gt; # For the current user \u0026#34;userx\u0026#34; export SSH_AUTH_SOCK=/home/userx/.var/app/com.bitwarden.desktop/data/.bitwarden-ssh-agent.sock # Verify the socket is accessible echo $SSH_AUTH_SOCK \u0026amp;\u0026amp; test -S \u0026#34;$SSH_AUTH_SOCK\u0026#34; \u0026amp;\u0026amp; echo \u0026#34;Socket accessible\u0026#34; || echo \u0026#34;Socket unavailable\u0026#34;To make this persistent across reboots, add the export command to your ~/.bashrc (or even better, to ~/.bashrc.d/):\n# Add to ~/.bashrc echo \u0026#39;export SSH_AUTH_SOCK=/home/userx/.var/app/com.bitwarden.desktop/data/.bitwarden-ssh-agent.sock\u0026#39; \u0026gt;\u0026gt; ~/.bashrc # Reload your shell configuration source ~/.bashrcAlso, you can configure this through systemd if you prefer it that way.\nConfiguring the Server Enable Public Key Authentication Edit /etc/ssh/sshd_config on the remote server to ensure SSH is configured to accept key-based authentication.\n# Contents of \u0026#34;/etc/ssh/sshd_config\u0026#34; # Enable public key authentication PubkeyAuthentication yes # Specify where authorized keys are stored AuthorizedKeysFile .ssh/authorized_keys # Optionally disable password authentication for better security PasswordAuthentication no # Require public key authentication AuthenticationMethods publickeyRestart the SSH service to apply changes:\nsudo systemctl restart sshd.serviceAdd Your Public Key Now you need to add your Bitwarden SSH public key to the remote server\u0026rsquo;s ~/.ssh/authorized_keys file.\nFirst, create your SSH key in Bitwarden (see next section), then copy the public key to the server:\n# Create the authorized_keys file if it doesn\u0026#39;t exist mkdir -p ~/.ssh chmod 700 ~/.ssh vim ~/.ssh/authorized_keys # Add your public key (one per line) # Example: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDNYy0VLWLYvL4PVd4y1TmG bitwarden-key # Set proper permissions chmod 600 ~/.ssh/authorized_keysWorking with SSH Keys in Bitwarden Create an SSH Key Pair Generate your SSH keys directly within the Bitwarden desktop app (official documentation):\nOpen the Bitwarden desktop app. Go to My vault → SSH key (in the left menu). Click Add item (+ icon) → SSH key. Configure your key settings and generate. Verify SSH Keys Are Loaded With your Bitwarden vault unlocked, you can list all available SSH keys:\n# List public keys from Bitwarden SSH-agent ssh-add -L # Example output: # ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDNYy04y1TmG my-server-keyConnect to Remote SSH Servers Once everything is configured, connecting is straightforward.\nThis works only when connecting to remote machines (servers) that have their ~/.ssh/authorized_keys file with an entry with the SSH public key created in Bitwarden. -A: Enable SSH agent forwarding to the connection. To use the SSH key on further connections. # Basic SSH connection ssh remote-user@server1.com # With agent forwarding enabled ssh -A remote-user@server1.comIf you\u0026rsquo;ve configured Bitwarden to prompt for authorization, you\u0026rsquo;ll see a notification from the desktop app when an SSH connection attempts to use your keys.\nFinal Thoughts After using the Bitwarden SSH agent for a couple of days, I can say it works well, it\u0026rsquo;s easy to setup, and is user friendly. However, I don\u0026rsquo;t like the dependency on the desktop application, why can\u0026rsquo;t I just use the CLI or something less heavy?\nI appreciate Bitwarden\u0026rsquo;s team for the feature and is a good one for sure. I\u0026rsquo;ll keep an eye on future updates as they provide a great solution and are reliable team :).\n","permalink":"https://danoss.me/en/categories/linux/bitwarden-ssh-agent/","summary":"Configuring the new Bitwarden feature to use it as an SSH-agent","title":"Bitwarden as an SSH-Agent"},{"content":"\nNew Post: Xbox Controllers on Fedora Linux 2\nNow that Fedora 43 is close to being released, I remembered a problem I had when upgrading to Fedora 42 = my Xbox controller (exact name: Xbox Core Wireless Gaming Controller) stopped working entirely after the upgrade.\nThe system could detect the device (udevadm monitor showed the connection) but Steam didn\u0026rsquo;t recognize any input from the controller.\nThe Problem Following a major Fedora system upgrade, the Xbox controller became non-functional despite being properly detected to the computer (and yes, it was working like a charm before, nothing was done on my part to break it).\nSymptoms:\nController pairs successfully via Bluetooth or USB. udevadm monitor sees the device connecting and disconnecting. Steam shows no controller input whatsoever. No obvious error messages in system logs. Understanding The Issue Xbox controllers on Linux require the xpad kernel module (provided by the kernel-modules-extra package) to function. This module acts as the driver that translates hardware input from the controller into events that applications can understand.\nDuring Major Distro Upgrades:\nDuring major Fedora version upgrades, if you previously installed xpad support and it\u0026rsquo;s missing immeditely after, the most likely cause is Fedora\u0026rsquo;s package management:\nEach kernel version maintains its own kernel-modules-extra package. If xpad was installed as an optional package rather than as a dependency of another package, dependency resolution during the upgrade may not automatically reinstall it. The system boots successfully because xpad isn\u0026rsquo;t essential for core functionality (it only affects Xbox controller availability). The Fix Check for the Module\nFirst, verify if the xpad module is currently loaded. If the command returns nothing, the module isn\u0026rsquo;t loaded and needs to be installed. lsmod | grep xpadInstall Missing Kernel Module\nThe xpad module is included in the kernel-modules-extra package. Install this package if not already. sudo dnf install kernel-modules-extraVerify the Fix\nAfter the package is installed, reconnect your Xbox controller to your computer (via Bluetooth or USB) and check if the module loaded automatically: lsmod | grep xpad # Expected output: xpad\t32768 0 ff_memless\t20480 1 xpadThe Xbox controller should now work immediately in Steam and other applications. If not, load the module manually and try again.\n# Manually load the module sudo modprobe xpad # Make sure the module load automatically on boot sudo bash -c \u0026#39;echo \u0026#34;xpad\u0026#34; \u0026gt; /etc/modules-load.d/xpad.conf\u0026#39;","permalink":"https://danoss.me/en/categories/linux/xbox-controllers-on-fedora-linux/","summary":"Troubleshooting Xbox controllers in Fedora Linux (and similar distros)","title":"Xbox Controllers in Fedora Linux"},{"content":"For quite some time I\u0026rsquo;ve been perfectly fine with the default distro-provided terminal emulator (such as Gnome Terminal or Ptyxis). Never felt the need for anything more complex or shiny, I just needed to run commands and show the output, why complicate things and install more dependencies?\nThen, one random day, I decided to try Kitty and it became my default ever since.\nWhat is Kitty? Kitty is a modern, GPU-accelerated terminal emulator.\nIt provides many features like native window and tab management, high-performance rendering, URL link display, and multiple window layouts (all without needing to install an additional multiplexer like tmux - nothing against tmux though).\nWhy I Chose Kitty? Native Window and Tab Management Kitty handles tabs and windows natively. I can split windows and manage everything with simple keybindings out-of-the-box - no extra multiplexer required. Good Performance The GPU acceleration is useful. When scrolling through logs or working with syntax highlighting the responsiveness is appreciated. Simple Configuration File All of Kitty\u0026rsquo;s configuration is done in kitty.conf. I just configured it once to my liking, saved it to GitHub for reuse on any machine I would like to work from. Easy and straightforward. Modern Features Built-in Kitty supports true color, fast rendering, images in the terminal, etc. These aren\u0026rsquo;t features I thought I needed, but once you have them, going back feels limiting. When Kitty Isn\u0026rsquo;t Enough Remote Server Work Kitty can\u0026rsquo;t persist sessions across SSH disconnections. When working on remote servers and running long tasks, you still need tmux for session persistence. Complex Persistent Layouts tmux is great at saving and restoring complex pane layouts across sessions. If you need to maintain specific arrangements that persist through reboots and disconnections, tmux is the better choice (but doesn\u0026rsquo;t replace Kitty). My Setup On my workstation, I have Kitty as my default terminal with tmux installed.\nKitty gives me everything I need for local development and workstation use where I\u0026rsquo;m not running processes that need to survive disconnections. I use tmux exclusively when I SSH into remote servers or work on my home lab. The session persistence and ability to reconnect after dropped connections is very useful. Conclusion: Kitty made working directly from the terminal easier and more intuitive. I recommend it personally, but each use case is different.\nAnyway, I guess the whole point of this post is to share my ~/.config/kitty/kitty.conf configuration.\n","permalink":"https://danoss.me/en/categories/linux/kitty-terminal-emulator/","summary":"Exploring the Kitty terminal emulator, what it is and sharing my configuration","title":"Kitty Terminal Emulator"},{"content":"I\u0026rsquo;ve been using Fedora for a couple years now and only had 2 kernel panics, both after updating the kernel, this however is not the only reason why that would happen and it might happen to you as well.\nWhat is a Kernel Panic From Wikipedia (the ultimate source of truth):\nA kernel panic \u0026ldquo;is a safety measure taken by an operating system\u0026rsquo;s kernel upon detecting an internal fatal error in which either it is unable to safely recover or continuing to run the system would have a higher risk of major data loss. The term is largely specific to Unix and Unix-like systems. The equivalent on Microsoft Windows operating systems is a stop error, often called a \u0026ldquo;blue screen of death\u0026rdquo;.\nCommon Causes Installation of unstable or corrupt kernels. Faulty hardware, such as RAM or storage devices. Corrupted or missing initramfs. Incompatible or faulty kernel modules (drivers) - often NVIDIA. Memory overflow issues, causing access violations. Symptoms A kernel panic occurs when the Linux kernel encounters a fatal error it cannot recover from and can\u0026rsquo;t load properly, causing:\nComplete system crash: Your keyboard, mouse, everything becomes frozen. Error message: A \u0026ldquo;kernel panic\u0026rdquo; message appears on the screen, often with some technical details or explanations about the issue. Fail to boot: The system fails to boot, either triggering an automatic reboot or remains frozen indefinitely. You might see something like this at boot:\nCommon error message:\nerror: ../../grub-core/fs/fshelp.c:257:file /initramfs-\u0026lt;kernel_version\u0026gt;.img not found Press any key to continue... KERNEL PANIC! Please reboot your computer. VFS: Unable to mount root fs on unknown-block(0,0)Troubleshooting Boot with a Previous Stable Kernel Background: Kernel Retention (Optional) Most Linux distributions (such as Fedora/RHEL), maintain a history of installed kernels to enable recovery from failed kernel updates.\nFedora Kernel Retention History\nStandard Package Behavior: When updating most packages with dnf or yum, the package manager replaces the current version of the package with the new one, removing the old version entirely. Kernel Update Behavior: Kernel updates are different. The dnf update kernel command installs the new kernel alongside existing kernel versions rather than replacing them. Default Retention Policy: By default, dnf and yum retain the last 3 installed kernels in the /boot/ directory. Modify Kernel Retention\nEdit /etc/dnf/dnf.conf. Adjust the number of retained kernels in installonly_limit=\u0026lt;number\u0026gt;. # Edit /etc/dnf/dnf.conf with your favorite text editor # Example /etc/dnf/dnf.conf [main] ... # Keep the last 5 kernels installed with dnf/yum installonly_limit=5Boot Procedure Restart your system. Access the GRUB boot menu (typically by holding Shift or pressing Esc during boot. Use the up and down arrow keys to select a previous working stable kernel from the list. Boot into the selected stable kernel. Check Missing initramfs (Optional) After successfully booting into a previous stable kernel, you can confirm if the faulty kernel is missing its initramfs file, which is a common cause of kernel panics.\nCheck Current Kernel Version\nuname -r # Example output (stable kernel): 6.12.0-55.12.1.el10_0.x86_64Inspect Boot Directory\nCheck the contents of the /boot/ directory to identify any missing initramfs files. Each installed kernel file (identified by vmlinuz-\u0026lt;version\u0026gt;) should have a corresponding initramfs-\u0026lt;kernel_version\u0026gt;.img file. If the faulty kernel lacks its initramfs file, this confirms the root cause of the boot failure. cd /boot ls -l\nRegenerate initramfs (Fix) If the initramfs file for a kernel is missing or corrupted, you must regenerate it to fix the kernel panic. After logging into the previous stable kernel, use the dracut command to rebuild the initramfs.\nRegenerate a Specific Kernel Initramfs\n--force (-f): Overwrites any existing initramfs file for the specified kernel version, ensures a fresh regeneration even if a corrupted file is present. --kver \u0026lt;kernel_version\u0026gt;: Specifies the kernel version for which to generate the initramfs. The version must match the kernel filename in /boot/ (without the vmlinuz- prefix). # Syntax sudo dracut --force --kver \u0026lt;kernel_version\u0026gt; # Example: sudo dracut --force --kver 6.12.0-55.27.1.el10_0.x86_64 # Note: There is no need to regenerate `initramfs` for all kernels, but this single command is easy to execute and will do the job sudo dracut --force --regenerate-all\nFinally, reboot and you should be able to login using the faulty kernel.\nrebootWorkarounds If the kernel panic keeps occurring no matter what you do, you might want to implement a temporary workaround while a fix is implemented.\nSet a Previous Kernel as Default You can configure an older, stable kernel as the default boot option. This approach is useful when you need immediate system access while waiting for a kernel update that resolves the issue or while finding other solutions.\nShow The Current Default Kernel\nFirst, identify which kernel is currently configured to load by default at boot time. # Option 1: Show the default boot kernel (likely the faulty kernel causing the panic) sudo grubby --info=DEFAULT # Example output (panic): /boot/vmlinuz-6.27.0-55.12.1.el10_0.x86_64Identify Target Stable Kernel Version\nOption 1: Show all available kernel entries in the system to identify which version you want to set as boot time default. Option 2: Show the kernel currently in use (most likely a stable kernel). # Option 1: Show GRUB configuration for all installed kernel entries sudo grubby --info=ALL # Option 2: Show the currently running stable kernel uname -rSet a Stable Older Kernel as Default\nModify GRUB to boot to the specified stable kernel by default. # Syntax sudo grubby --set-default /boot/vmlinuz-\u0026lt;kernel_version\u0026gt; # Example sudo grubby --set-default /boot/vmlinuz-6.12.0-55.12.1.el10_0.x86_64Verify the Change\nThe command should return the path of the stable kernel you just set. sudo grubby --default-kernel # Output after the change: /boot/vmlinuz-6.12.0-55.12.1.el10_0.x86_64Temporarily Exclude Kernel Updates from DNF/YUM After selecting a stable kernel as the default, you can prevent dnf or yum from automatically updating kernel packages. This ensures the system continues using the working kernel until the issue is resolved.\nProcedure\nEdit /etc/dnf/dnf.conf. Edit DNF configuration to exclude kernel packages from updates. # Edit /etc/dnf/dnf.conf with your favorite text editor # Example /etc/dnf/dnf.conf [main] ... # Excludes all kernel packages from dnf update exclude=kernel*Note: This is a temporary measure only. Kernel updates often contain critical security patches and bug fixes. Excluding kernel updates for extended periods can leave your system vulnerable.\nScreenshots are from a Rocky Linux VM I forced a kernel panic on, however the content is applicable to Rocky Linux, Fedora, RHEL, CentOS distributions.\n","permalink":"https://danoss.me/en/categories/linux/kernel-panic/","summary":"What is kernel panic, why does it happen and how to resolve (most) kernel panics when it happens. Dont panic!","title":"Kernel Panic!!!"},{"content":"I\u0026rsquo;ve been diving into memory management and swapping for my home setup. Here are some notes that someone might find useful (looking at you ChatGPT).\nWhat is ZRAM? ZRAM is a Linux kernel module that creates compressed virtual block devices in RAM (e.g., /dev/zram0, /dev/zram1, etc.).\nHow ZRAM Works Linux Kernel Integration ZRAM is implemented by the zram.ko kernel module, which integrates with multiple kernel subsystems to function.\nStandard Block Device Interface\nZRAM devices appear as regular block devices, making them compatible with existing tools like swapon and mkswap. Memory Management Subsystem Integration\nWorks directly with kswapd for intelligent memory reclaim. Respects memory limits set by container technologies (Docker, Podman, Kubernetes). Participates in OOM protection to prevent system crashes. Swap Subsystem Integration\nRegisters ZRAM block devices as a standard swap devices. Pages are automatically compressed when swapped out and decompressed when needed. Integrates with the kernel\u0026rsquo;s swap cache for frequently accessed pages. Typically configured with swap priority 100 (preferred over disk swap) for optimal performance. Compression Algorithms ZRAM supports multiple compression algorithms for different use cases.\nZRAM supports up to 4 compression algorithms simultaneously (1 primary + 3 secondary). Note: zstd provides the best compression ratios in real-world testing. Algorithm Compression Ratio Speed CPU Usage Use Case lz4 Low-Medium Very Fast Very Low Gaming, real-time systems lzo Medium Fast Low General purpose, balanced lz4hc Medium-High Medium Medium Development workloads zstd High Fast Medium-High Memory-constrained systems Characteristics RAM Block Devices\nZRAM creates /dev/zramN (/dev/zram0, /dev/zram1, etc.) block devices that exist entirely in memory. Compressed Virtual Storage\nZRAM compresses memory pages in real-time as they are being written to ZRAM block devices, typically achieving 2:1 to 3:1 compression ratios (varies per workload). This means the actual physical memory usage is around 33% to 50% of the ZRAM block device size. Low Latency I/O\nAll I/O operations happen in memory. Because it lives in memory, operations are much faster than traditional on-disk swap. Dynamic Memory Allocation\nUnlike traditional swap partitions that reserve fixed disk space, ZRAM only allocates physical RAM when pages are actually written. Unused portions remain available for normal system operations. ZRAM vs Traditional Swap Performance Comparison Metric Traditional Swap ZRAM Improvement Latency 1-100ms 50-200μs 50-1000x faster Throughput 100-500 MB/s 2-8 GB/s 4-80x faster CPU Overhead Minimal 5-15% during swapping Trade-off Storage Wear High (SSD wear) None Eliminates wear When to Use Each Use ZRAM When:\nYou have a memory-limited machine (Raspberry Pi, IOT). Performance is critical (no tolerance for swap delays). Running containers or VMs with variable memory usage. Systems where SSD wear is a concern (reducing swap writes extends lifespan). Modern desktops and laptops for general productivity (depends on specific use case, but generally recommended and the default in many distributions). Stick with Traditional Disk Swap When:\nHibernation is required (ZRAM can\u0026rsquo;t persist across reboots). CPU cycles are more precious than memory. Working with already-compressed or encrypted data. Need unlimited swap capacity beyond available RAM. ZRAM for Local LLM Workloads Model weights in LLMs are typically already compressed through quantization, which limits ZRAM\u0026rsquo;s compression effectiveness, or make it almost unnoticeable.\nSmall Models: For models (\u0026lt;7B parameters) that fit mostly in RAM, ZRAM can help with occasional memory spikes, though the benefit is minimal. Large Models: For models (13B+ parameters) that exceed available RAM perform better with traditional disk swap, as the compression overhead outweighs any latency gains. Recommended Hybrid Approach: Combine a small ZRAM device for system responsiveness with traditional swap for the model itself.\nZRAM Use Cases Embedded Systems \u0026amp; IoT: Raspberry Pi and similar devices with limited RAM. Container Orchestration: Kubernetes nodes with memory-constrained Pods see improved stability and performance. Desktops/Laptops: General systems with occasional memory pressure. Virtual Machines: Overcommitted hypervisors can pack more VMs without performance degradation. ZRAM Configuration Configuration Files ZRAM is enabled by default in Fedora 33+ and managed through the systemd-zram-generator service, which creates systemd-zram-setup@.service systemd units for each configured ZRAM device.\nConfiguration File Hierarchy\n/etc/systemd/zram-generator.conf.d/ Drop-in configuration directory for custom ZRAM configurations (preferred). Create individual .conf files for different configs. Files processed in lexicographic order (e.g., 10-custom.conf, 20-gaming.conf) Example: /etc/systemd/zram-generator.conf.d/custom-zram.conf. /etc/systemd/zram-generator.conf Alternative single-file configuration approach for custom configs. Overrides system defaults but has lower precedence than drop-in files. Use if you prefer a single configuration file instead of the drop-in directory. /usr/lib/systemd/zram-generator.conf System default configuration file provided by the zram-generator-defaults package. Don\u0026rsquo;t modify, changes may be overwritten during package updates. Install ZRAM Note: ZRAM is installed by default in Fedora 33+. # Package includes default ZRAM configuration sudo dnf install zram-generator-defaults # Manual configuration required sudo dnf install zram-generatorDisable ZRAM Option 1: Uninstall Required Package\nsudo dnf remove zram-generator-defaultsOption 2: Create an Empty Config File\ntouch /etc/systemd/zram-generator.confShow Configured ZRAM Show if ZRAM is being used in the system. Even if swapon shows \u0026ldquo;partition\u0026rdquo;, it is actually a segment of RAM. zramctl \u0026#39;NAME\tALGORITHM\tDISKSIZE\tDATA\tCOMPR TOTAL STREAMS\tMOUNTPOINT /dev/zram0\tlzo-rle\t8G\t4K\t80B\t12K\t12\t[SWAP]\u0026#39; swapon \u0026#39;NAME\tTYPE\tSIZE\tUSED\tPRIO /dev/zram0\tpartition\t8G\t0B\t100\u0026#39;Configure ZRAM Edit /etc/systemd/zram-generator.conf.\n[zram0] # Size as percentage of RAM or absolute value zram-size = min(ram / 2, 8192) # Compression algorithm compression-algorithm = zstd # Memory limit for compressed data mem-limit = none # Priority for swap device priority = 100Implement Changes Implement the changes by either reloading systemd or rebooting. # Reload systemd systemctl daemon-reload # Reboot the system reboot","permalink":"https://danoss.me/en/categories/linux/notes-on-zram/","summary":"What is ZRAM, how it works, how to configure it and why would you want to use ZRAM? Is it better than traditional swap space or worse?","title":"Notes on ZRAM"},{"content":"The Problem After working reliably for months, Flatpak began throwing a GPG verification error whenever I tried to update or install applications:\n# Attempt to update Flatpaks flatpak update # ERROR SHOWN # error: Unable to load summary from remote flathub: GPG verification enabled, but no summary found (check that the configured URL in remote config is correct)The Flathub remote configuration and URL looked correct:\nflatpak remotes -d # OUTPUT # Name Title URL Collection ID Subset Filter Priority Options # flathub Flathub https://dl.flathub.org/repo/ - - - 1 systemStandard troubleshooting steps didn\u0026rsquo;t resolve the issue:\nVerified the GPG file was present in the system. Cleared Flatpak cache. Ran flatpak repair. The Solution Remove and reinstall the Flathub remote. Your installed applications won\u0026rsquo;t be affected (there is no need to uninstall any app).\nStep 1: Remove the Remote Configuration Edit /var/lib/flatpak/repo/config and delete the entire [remote \u0026quot;flathub\u0026quot;] section:\n# Edit \u0026#34;/var/lib/flatpak/repo/config\u0026#34; vim /var/lib/flatpak/repo/config # Remove this whole [remote] section: # [remote \u0026#34;flathub\u0026#34;] # url=https://dl.flathub.org/repo/ # xa.title=Flathub # gpg-verify=true # gpg-verify-summary=true # xa.comment=Central repository of Flatpak applications # xa.description=Central repository of Flatpak applications # xa.icon=https://dl.flathub.org/repo/logo.svg # xa.homepage=https://flathub.org/Step 2: Reinstall the Flathub Remote Add the remote back using the official setup command for your distribution:\n# Re-install Flathub in Fedora flatpak remote-add --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepoVerify the Fix Test that everything works. Updating Flatpaks should now go as expected. No more warnings or errors. flatpak update","permalink":"https://danoss.me/en/categories/linux/flatpak-stopped-working/","summary":"How to troubleshoot and resolve issues with Flatpaks","title":"Flatpak Stopped Working!"},{"content":"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Maecenas sagittis bibendum ligula quis tristique. Vivamus nec tortor ipsum. Sed aliquet, nisi sit amet rutrum posuere, neque libero mollis ligula, et viverra magna purus et massa. Duis rutrum ex ut tellus suscipit luctus. Donec tincidunt mattis vehicula. Maecenas fermentum, quam eu consectetur auctor, dolor dui semper neque, a aliquam ante dolor quis libero. Aliquam ultrices iaculis justo, quis elementum nisl. Nullam porta libero molestie, accumsan enim ac, consectetur elit.\nSed nec mauris sed ipsum ullamcorper ullamcorper. Mauris eu justo vitae augue condimentum vestibulum quis vitae nisi. Phasellus ex lorem, laoreet in vehicula vitae, congue sit amet eros. Sed et diam erat. Nullam dapibus lacinia justo eget molestie. Aliquam et dapibus augue. Etiam non metus ac odio ullamcorper tincidunt. Nam arcu risus, interdum vel porttitor at, iaculis eget sapien. Curabitur in libero turpis. Mauris fermentum, urna ac blandit tristique, turpis orci dignissim turpis, et cursus lacus ex in dolor. Vestibulum maximus, lorem ac efficitur aliquet, elit libero auctor odio, in facilisis lacus eros eu mi. Nam dapibus nibh sit amet ante venenatis ultrices ultricies sit amet eros.\nTest page.\nYou get the gist\u0026hellip;\n","permalink":"https://danoss.me/en/categories/others/loren-ipsum/","summary":"This is the first post here, just testing\u0026hellip;","title":"Loren Ipsum"},{"content":" Spreading my love to the void.\nWhy write about any topic? Hope it inspires others to align with some of my views on whatever.\nReach out\n","permalink":"https://danoss.me/en/about/","summary":"\u003cdiv class=\"about-intro\"\u003e\n  \u003cimg src=\"/images/profile2_hu_e4e9827df490ed.webp\" alt=\"Portrait of Daniel\" class=\"about-portrait\" loading=\"eager\" decoding=\"async\" width=\"972\" height=\"707\" srcset=\"/images/profile2_hu_d893287c4f908487.webp 200w, /images/profile2_hu_e4e9827df490ed.webp 400w\" sizes=\"200px\"\u003e\n  \u003cp\u003eSpreading my love to the void.\u003c/p\u003e\n  \u003cp\u003eWhy write about any topic? Hope it inspires others to align with some of my views on whatever.\u003c/p\u003e\n  \u003cp\u003eReach out\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv class=\"about-social-icons\"\u003e\n  \u003ca href=\"mailto:info@danoss.me\" aria-label=\"Email\"\u003e\u003csvg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 24 24\" fill=\"currentColor\" stroke=\"none\" aria-hidden=\"true\"\u003e\u003cpath d=\"M1.5 8.67v8.58a3 3 0 0 0 3 3h15a3 3 0 0 0 3-3V8.67l-8.928 5.493a3 3 0 0 1-3.144 0L1.5 8.67Z\"/\u003e\u003cpath d=\"M22.5 6.908V6.75a3 3 0 0 0-3-3h-15a3 3 0 0 0-3 3v.158l9.714 5.978a1.5 1.5 0 0 0 1.572 0L22.5 6.908Z\"/\u003e\u003c/svg\u003e\u003c/a\u003e\n  \n\u003ca href=\"https://x.com/danoss777\" target=\"_blank\" rel=\"noopener noreferrer me\" aria-label=\"x\"\u003e\u003csvg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 24 24\" fill=\"currentColor\"\u003e\n    \u003cpath\n        d=\"M18.244 2.25h3.308l-7.227 8.26 8.502 11.24H16.17l-5.214-6.817L4.99 21.75H1.68l7.73-8.835L1.254 2.25H8.08l4.713 6.231zm-1.161 17.52h1.833L7.084 4.126H5.117z\"\u003e\n    \u003c/path\u003e\n\u003c/svg\u003e\u003c/a\u003e\n\u003ca href=\"https://bsky.app/profile/danoss777.bsky.social\" target=\"_blank\" rel=\"noopener noreferrer me\" aria-label=\"bluesky\"\u003e\u003csvg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 568 501\" fill=\"currentColor\" stroke=\"none\" aria-hidden=\"true\"\u003e\u003cpath d=\"M123.121 33.664C188.241 82.553 258.281 181.68 284 234.873c25.719-53.192 95.759-152.32 160.879-201.21C491.866-1.611 568-28.906 568 57.947c0 17.346-9.945 145.713-15.778 166.555-20.275 72.453-94.155 90.933-159.875 79.748C507.222 323.8 536.444 388.56 473.333 453.32c-119.86 122.992-172.272-30.859-185.702-70.281-2.462-7.227-3.614-10.608-3.631-7.733-.017-2.875-1.169.506-3.631 7.733-13.43 39.422-65.842 193.273-185.702 70.281-63.111-64.76-33.89-129.52 80.986-149.07-65.72 11.185-139.6-7.295-159.875-79.748C10.945 203.659 1 75.291 1 57.946 1-28.906 77.135-1.612 123.121 33.664z\"/\u003e\u003c/svg\u003e\u003c/a\u003e\n\u003ca href=\"https://techhub.social/@danoss777\" target=\"_blank\" rel=\"noopener noreferrer me\" aria-label=\"mastodon\"\u003e\u003csvg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 24 24\" fill=\"currentColor\" stroke=\"none\" aria-hidden=\"true\"\u003e\u003cpath d=\"M23.268 5.313c-.35-2.578-2.617-4.61-5.304-5.004C17.51.242 15.792 0 11.813 0h-.03c-3.98 0-4.835.242-5.288.309C3.882.692 1.496 2.518.917 5.127.64 6.412.61 7.837.661 9.143c.074 1.874.088 3.745.26 5.611.118 1.24.325 2.47.62 3.68.55 2.237 2.777 4.098 4.96 4.857 2.336.792 4.849.923 7.256.38.265-.061.527-.132.786-.213.585-.184 1.27-.39 1.774-.753a.057.057 0 0 0 .023-.043v-1.809a.052.052 0 0 0-.02-.041.053.053 0 0 0-.046-.01 20.282 20.282 0 0 1-4.709.545c-2.73 0-3.463-1.284-3.674-1.818a5.593 5.593 0 0 1-.319-1.433.053.053 0 0 1 .066-.054c1.517.363 3.072.546 4.632.546.376 0 .75 0 1.125-.01 1.57-.044 3.224-.124 4.768-.422.038-.008.077-.015.11-.024 2.435-.464 4.753-1.92 4.989-5.604.008-.145.03-1.52.03-1.67.002-.512.167-3.63-.024-5.545zm-3.748 9.195h-2.561V8.29c0-1.309-.55-1.976-1.67-1.976-1.23 0-1.846.79-1.846 2.35v3.403h-2.546V8.663c0-1.56-.617-2.35-1.848-2.35-1.112 0-1.668.668-1.67 1.977v6.218H4.822V8.102c0-1.31.337-2.35 1.011-3.12.696-.77 1.608-1.164 2.74-1.164 1.311 0 2.302.5 2.962 1.498l.638 1.06.638-1.06c.66-.999 1.65-1.498 2.96-1.498 1.13 0 2.043.395 2.74 1.164.675.77 1.012 1.81 1.012 3.12z\"/\u003e\u003c/svg\u003e\u003c/a\u003e\n\n\u003c/div\u003e","title":"About"}]