Practical Hash Tips
This page focuses on using hashes safely in real projects: collision risk, when to use HMAC or signatures, password hashing, and avoiding common pitfalls in encoding and parsing.
🧭 What This Covers
Overview
- ✅ Collision risk intuition and cautions when truncating hashes
- ✅ Checksum (CRC) vs cryptographic hash and where each fits
- ✅ HMAC vs digital signatures vs password hashing (KDF)
- ✅ Encoding/normalization pitfalls that change the hash
- ✅ Streaming large files and content-addressable examples
⚠️ Collisions and Truncation
These counts give about a 50% chance of at least one collision among independent, uniformly distributed N-bit values. They are not attack costs for weakened algorithms.
Bit-length intuition
| Bits | Count at about 50% |
|---|---|
| 64 | 5.06 × 10^9 |
| 128 | 2.17 × 10^19 |
| 160 | 1.42 × 10^24 |
| 256 | 4.01 × 10^38 |
When truncating
- ✂️ The approximation is n ≈ √(2 ln 2) × 2^(N/2). Truncation reduces the margin against both accidental collisions and collision search.
- ✂️ Keep the full value when shortening identifiers for display, and define how collisions are handled.
- ✂️ Finding any collision differs from finding an input for a specified digest. A hash is neither a public key nor encryption.
🧮 Checksum vs Cryptographic Hash
CRC/Adler detect random errors; they do not resist intentional tampering.
Choose appropriately
- 🔍 Use CRC-style checksums for accidental corruption, not adversarial forgery.
- 🔍 Even with SHA-256, obtain the expected digest through a trusted channel. Matching values do not help if an attacker replaces both file and reference.
- 🔍 Use a suitable authentication mechanism, such as HMAC or a signature verified with a trusted key.
🧰 HMAC / Signatures / Password Hashing
Who holds the secret and what you protect determines the tool.
Roles
- 🧠 HMAC authenticates with a shared secret. All key holders can create tags; it does not encrypt the content.
- 🧠 Verify signatures using a trusted public key. Apply canonicalization only as defined by the protocol.
- 🧠 Use a dedicated salted password-hashing library. Argon2id and scrypt support memory-cost settings; PBKDF2 controls computational cost through iterations. These are not all memory-hard schemes.
Migration tips
- 🔧 On login, rehash with the new scheme to migrate gradually.
- 🔧 A pepper (app-held secret) adds resilience but needs rotation planning.
🧾 Encoding and Normalization Pitfalls
Distinguish changes to input bytes from changes in how the same digest is displayed.
Common gotchas
- 📌 Hex case, `0x` prefixes, and separators create different strings.
- 📌 Base64 and Base64url strings can encode the same digest bytes. A different representation does not necessarily mean a different digest.
- 📌 Input newline (LF/CRLF) or Unicode normalization (NFC/NFD) differences yield different hashes.
Mitigations
- 🛠️ Normalize only when required by the agreed specification. Do not independently transform downloaded file bytes or signature inputs.
- 🛠️ Align on the encoding (hex/BASE64URL) when exchanging hashes externally.
⏩ Streaming Hashes and Commands
Feed chunks in order into the same hash state to obtain the same digest as hashing the complete input. This differs from concatenating separate chunk digests.
Practical points
- 🚚 In PHP, update the same context with hash_update and finish with hash_final.
- 🚚 For manifests, specify filenames, algorithms and expected digests, and protect the manifest's delivery.
- 🚚 Partial verification needs trusted per-chunk references or a verification tree. A full-file digest alone cannot verify an isolated chunk.
📦 Content-Addressable Examples
Addressing by content improves reproducibility and caching.
Examples
- 🧱 Git object IDs cover a header and content, not just raw file bytes. SHA-1 or SHA-256 depends on the repository format.
- 🧱 An IPFS CID contains format information as well as a hash. Chunking and DAG structure can affect it; it need not equal a hash of the complete raw file.
❓ FAQ
Q. Is MD5 or SHA-1 acceptable?
- A. Not for collision resistance. Restrict to legacy/non-safety contexts and plan replacement.
Q. How many bits are safe?
- A longer hash does not provide confidentiality. Use encryption for secrecy; choose hash length and algorithm according to collision resistance, preimage resistance and protocol requirements.
Specifications and related guides
Argon2 / PBKDF2 / Base64 / Git / IPFS CID / Integrity checks