Where unit conversions lose precision

Unit conversion looks like the safest arithmetic there is. It is not quite, and the failures are interesting: two different definitions sharing one abbreviation, binary arithmetic that cannot represent common decimals, and results printed to more digits than the measurement ever justified.

The kilobyte problem

This is the conversion ambiguity people meet most often, usually while wondering where a fifth of their hard drive went.

• Decimal: 1 KB = 1000 bytes. Used by storage manufacturers, networking equipment and the international standard.
• Binary: 1 KB = 1024 bytes. Used by most operating systems when reporting file and volume sizes.

Both write it as KB. The gap compounds with each step: 2.4% at kilobytes, 4.9% at megabytes, 7.4% at gigabytes and 10% at terabytes.

A drive sold as 1 TB
  = 1,000,000,000,000 bytes (decimal, as advertised)
  = 931.32 GB when divided by 1024 three times (binary, as reported)

Nothing has been lost. The manufacturer counted in one system and your operating system displays in another. The unambiguous binary names — KiB, MiB, GiB — exist precisely to end this argument, and remain rare outside technical documentation. The digital storage converter labels both conventions rather than picking one silently.

Bits and bytes

The related trap, and the reason people believe their broadband is underperforming. Connections are sold in megabits per second, and downloads are reported in megabytes per second. There are eight bits in a byte, so a 100 Mbps connection delivers at most 12.5 MB/s. Seeing 11 MB/s means the connection is working properly.

The capitalisation is the only clue: lowercase b is bits, uppercase B is bytes. Mbps and MB/s differ by a factor of eight.

Floating point, and why 0.1 + 0.2 is not 0.3

Computers store fractions in binary. Just as one third cannot be written exactly in decimal, one tenth cannot be written exactly in binary. It is stored as the nearest representable value, and the tiny difference surfaces when values are combined.

0.1 + 0.2          → 0.30000000000000004
1.1 * 3            → 3.3000000000000003

Every calculator, spreadsheet and programming language using standard double-precision arithmetic behaves this way. It is not a bug in the conversion factor. For everyday conversions the error is around the sixteenth significant digit and irrelevant; it matters when values are accumulated many times, which is why financial systems store money as integer minor units rather than as decimals.

Significant figures: the error people actually make

The most common precision mistake has nothing to do with the computer. It is converting a rough measurement and keeping every digit the conversion produced.

Measured:   "about 3 metres" (one significant figure)
Converted:  9.84251968503937 feet
Justified:  about 10 feet

Fourteen decimal places imply a measurement accurate to a fraction of a micrometre. The rule is that a converted value carries no more significant figures than the input did. A wall measured to the nearest centimetre is not suddenly known to the nearest thousandth of an inch.

Displayed digits are a formatting choice, not a claim about accuracy. Round to what you actually measured.

Where conversions are exact, and where they are not

Some factors are exact by international definition:

• 1 inch = 25.4 mm exactly
• 1 pound = 0.45359237 kg exactly
• 1 nautical mile = 1852 m exactly

Others are approximations that are marked as such:

• A month is not a fixed length. Using an average of 30.44 days expresses a duration and is wrong for calendar arithmetic — for that, use the date calculator.
• Mach depends on air temperature, so a fixed sea-level value is only ever an approximation.
• Horsepower comes in mechanical and metric flavours that differ by about 1.4%, which is why the same engine is quoted differently in different markets.

Same word, different quantity

Several unit names mean different amounts depending on where you are, and no converter can tell which one you meant:

• A US gallon is 3.785 litres; an imperial gallon is 4.546 — about 20% larger.
• A US pint is 16 fluid ounces; an imperial pint is 20.
• A fluid ounce measures volume; an ounce measures mass. They are not convertible without knowing the substance.
• A troy ounce, used for precious metals, is about 10% heavier than the avoirdupois ounce used for everything else.
• A US short ton is 2000 lb, a UK long ton 2240 lb, and a metric tonne 1000 kg.

The volume converter uses US customary definitions and says so, which matters if you are working from a British recipe.

Practical rules

1. Know which definition your source used before converting.
2. Round the result to the precision of the input, not of the display.
3. Convert once, at the end. Chaining conversions accumulates rounding error.
4. For anything with real consequences — dosage, engineering, structural loads — verify against a second source. Unit errors have caused spacecraft losses and aircraft fuel emergencies, and in every case the arithmetic was fine and the assumption was not.