Koha Solutions

Cataloging & MARC

Broken characters (�, é) in Koha after a MARC import

Diamonds with question marks, or é shown as é, mean the file was staged under the wrong character encoding. The characters cannot be repaired in Koha; undo the batch and import again with the right encoding.

By Babar Junaid

Published 2026-11-19 · Tested against Koha 26.05

Characters shown as �, or é shown as é, mean the MARC file was staged under the wrong character encoding. Koha's staging screen assumes UTF-8 and reports success either way: the only sign is a warning in the server log. A MARC-8 or Windows file staged that way has every accented, Urdu or Arabic letter replaced. Revert the import batch, convert the file to UTF-8 (or pick its real encoding when staging) and import it again.

The damaged records cannot be repaired in place

When the declared encoding is wrong, Koha replaces each byte it cannot read with � before saving. The original letter is not stored anywhere. Do not try to fix the records by search and replace: go back to the source file.
  • Keep the original file exactly as you received it.
  • If staff have edited the imported records since, note which ones before you revert the batch.

Step 1: Tell the two patterns apart

What you seeWhat happened
� next to an unaccented letter: Garc�ia M�arquezA MARC-8 file staged as UTF-8. MARC-8 stores the accent as its own byte before the letter, so the letter survives and the accent becomes �. Stage it as MARC 8
� in place of the letter: Caf� cr�me, or whole Urdu and Arabic words as � � �A single-byte file (Latin-1, a Windows code page such as Windows-1256) staged as UTF-8. The letter itself is gone. Convert it (step 4)
é, ü, ال and similar pairsA file that was already UTF-8, staged as MARC 8 or ISO 8859-1, so it was converted a second time
Only some records brokenThe file mixes encodings, usually several exports joined together

Step 2: Undo the import

  1. 1Go to Cataloging → Manage staged records.
  2. 2Open the batch you imported.
  3. 3Click Undo import into catalog. Records the batch created are deleted; records it overlaid are put back as they were.

Step 3: Find out what encoding the file really is

Terminal
$ yaz-marcdump records.mrc | head -n 300155nam  2200061 a 4500001 kb40-1100 1  $a Garc�ia M�arquez, Gabriel.
Response hidden (3 lines)
Showing the response of this command only

Response from a live run on Koha 26.05

The first line printed is the leader. Its 10th character (position 09) is a when the file claims UTF-8 and a space when it claims MARC-8, as here (nam 22: two spaces). Do not trust it on its own; many systems set it wrongly. Look at a record you know: if its accented or Urdu letters print correctly in a UTF-8 terminal, the file is UTF-8. This one is MARC-8, and it shows the MARC-8 pattern: the base letter survives and the accent beside it turns into � (Garc�ia, not Garc�a).

Files exported from older desktop systems in Urdu, Arabic or Persian are often in a Windows code page such as Windows-1256. There every letter of the script prints as �, for example 245 00 $a ����� ����. Koha's staging screen has no option for it, so those files must be converted first (step 4).

If the records came from a service you export from regularly (OCLC, a vendor portal, another library system), check its export setting too and set it to UTF-8, so the next file arrives right and needs no conversion.

Step 4: Convert the file to UTF-8

yaz-marcdump is installed with Koha. Give it the real source encoding with -f, the target with -t, binary MARC out with -o marc, and -l 9=97 to set leader position 09 to a (97 is the code for the letter a):

Terminal
$ yaz-marcdump -f MARC-8 -t UTF-8 -o marc -l 9=97 records.mrc > records-utf8.mrc$ yaz-marcdump records-utf8.mrc | head -n 300160nam a2200061 a 4500001 kb40-1100 1  $a García Márquez, Gabriel.
Response hidden (3 lines)
Showing the response of this command only

Response from a live run on Koha 26.05

The conversion prints nothing; check the new file, as the second command does. Position 09 of the leader is now a and the letters read correctly. Converting from MARC-8, yaz-marcdump writes each accented letter as two characters, the letter and a separate accent. It looks the same, and Koha joins them back into one when it stages the file, so nothing more is needed. For a Latin-1 file use -f ISO-8859-1. For a Windows Arabic-script file use -f CP1256:

Terminal
$ yaz-marcdump -f CP1256 -t UTF-8 -o marc -l 9=97 urdu.mrc > urdu-utf8.mrc$ yaz-marcdump urdu-utf8.mrc00081nam a2200049 a 4500001 kb40-2245 00 $a ديوان غالب
Response hidden (3 lines)
Showing the response of this command only

Response from a live run on Koha 26.05

One thing to know for Urdu: Windows-1256 has no Urdu ی (U+06CC), so a file in that code page always stored the Arabic ي (U+064A) in its place, and the converted record keeps it. The text reads correctly, but it is not the letter an Urdu keyboard types. After importing, search for a title that contains it to see whether your catalog finds it. Check a few more records with yaz-marcdump records-utf8.mrc | less before importing.

Step 5: Stage it again with the right setting

Stage records-utf8.mrc in Cataloging → Stage records for import with Character encoding left at UTF-8. If you did not convert the file, choose its real encoding instead: MARC 8 for most older MARC21 exports, ISO 8859-1 for Latin-1. Before you import, open one record from the staged batch preview and check its title.

Why this happens

Koha stores everything as UTF-8 and converts what it imports. To convert, it has to know what the file was, and the file cannot say so reliably. MARC21 has one flag for it, leader position 09, and so many systems have set it wrongly over the years that Koha's own code ignores it. The staging screen therefore asks you, and defaults to UTF-8.

When the answer is wrong, Koha has two bad choices: refuse the record or keep it with the unreadable bytes replaced. It keeps it, replacing each unreadable byte with the Unicode replacement character, so the import "succeeds" and the damage shows up later at the OPAC. It is not only cosmetic: a title or name with � in it no longer matches what readers type, so the record drops out of search, sorts in the wrong place and fails to match its authority record. The reverse mistake, converting a file that was already UTF-8, gives the é pairs: the conversion has run twice.

Troubleshooting

ProblemWhat to check
Letters right in the staff interface, wrong in exported filesThe program opening the export, often Excel, is guessing the encoding. Open it as UTF-8
Right in Koha, wrong in a Z39.50 record you copiedThe target server's encoding is set wrongly: see adding a Z39.50 server
Urdu or Arabic letters right but in the wrong orderA display direction problem, not encoding: the record's script and the OPAC language settings
Records you just imported cannot be found by searchThe index, not the characters: rebuild the Zebra index

The full staging and matching workflow is in importing MARC records into Koha, and the queue that runs the import is explained in background jobs stuck at New.

Tested on live Koha 26.05.05

Passed all 13 checks · Read the run log

Koha release
26.05.05
System
Ubuntu 24.04
Setup
A fresh Ubuntu 24.04 server reached over SSH as a sudo user, Koha installed from the Koha repository as our installation guide leaves it.
Source read at
Koha commit d3fc229
Earlier live checks (1)
  • , Koha 26.05.05 (commit d3fc229): all 11 claims held. Read the run log

In the Koha manual (26.05)

Verified against the Koha source (5 files)

The label under each command says where its response came from; a command with no label was not run live. How we test and which standards we follow.

Share this article

Characters still broken after re-importing? Send us a sample record and we will convert the whole file for you.

Want your staff trained on this? We teach it hands on with a live Koha, online or at your library. Ask about staff training