Recent Changes Printable View Page History Edit Page

Main.MARCUpload History

Hide minor edits - Show changes to markup

November 05, 2016, at 09:14 AM by 127.0.0.1 -
March 17, 2014, at 11:43 PM by Hans van der Weij - Update MarcEdit link
Changed line 39 from:

MARC data is not plain text data. Many programs allow you to edit MARC data in a plain text format, but that text format can not be imported directly into OpenBiblio. You will need to convert the records into actual MARC transmission format (Z39.2) before import. MarcEdit is an excellent, free program for Windows that can help you with this.

to:

MARC data is not plain text data. Many programs allow you to edit MARC data in a plain text format, but that text format can not be imported directly into OpenBiblio. You will need to convert the records into actual MARC transmission format (Z39.2) before import. MarcEdit is an excellent, free program for Windows that can help you with this.

February 26, 2012, at 04:47 PM by Hans van der Weij - update for 0.7.1
Changed lines 1-3 from:

In short

In the staff interface, go to the Cataloging tab, then click on 'Upload Marc Data' in the sidebar. You'll get a form where you can upload a file of MARC data. The first time you upload, you should probably leave the 'Test Load' setting at true so you can see what OpenBiblio thinks about your MARC file, without messing with the database. After that, go back to the upload screen and change the setting to false. Make sure that the collection and material type values are set the way you want them, and then upload the file again. You should get a message telling you how many records were imported. After the import, you'll need to find the imported records, set the call number (unless you use a patch to set it for you, see Patch:1118356), and add copies.

to:

In short

In the staff interface, go to the Cataloging tab, then click on 'Upload Marc Data' in the sidebar. You'll get a form where you can upload a file of MARC data. The first time you upload, you should probably leave the 'Test Load' setting at true so you can see what OpenBiblio thinks about your MARC file, without messing with the database. After that, go back to the upload screen and change the setting to false. Make sure that the collection and material type values are set the way you want them, and then upload the file again. You should get a message telling you how many records were imported. After the import, you'll need to find the imported records, set the call number, and add copies. This can be automated by using SQL commands.

(Patch:1118356 sets the call number automatically on import. This patch does not work in recent versions.)

Changed lines 8-9 from:

Important

to:

Important

Changed lines 12-18 from:

0.5.0, 0.6.0, etc.

The database table layout of these versions is a predecessor of the 1.0 layout and does not allow to import all of the MARC record format. Data loss:

  • Control fields.
  • Indicators.
  • Excluded fields (import_usmarc_excludes.php configures marc fields that are excluded from import).

When the input file is large, the script may not finish and no records are imported at all. Options to fix that:

to:

Prevent partial import / time out

Use 'Test Load' to have an indication if the import file isn't too large. There is no automated way to resume import at the point where it stopped. You can try to resume import after manually removing the already imported records from the import file.

Tips for preventing script time out:

Changed lines 22-27 from:

1.0-wip

Allows to import larger files, but it's still a good idea to split large import files. If the process times out, import is performed partially. You can try to resume import after manually removing the already imported records from the import file.

Pre-processing

to:

0.5.0, 0.6.0, etc.

The database table layout of these versions is a predecessor of the 1.0 layout and does not allow to import all of the MARC record format. Data loss:

  • Control fields.
  • Indicators.
  • Excluded fields (import_usmarc_excludes.php configures marc fields that are excluded from import).

When the input file is large, the script may not finish and no records are imported at all.

0.7.1

Equal to above, but if the process times out, import is performed partially.

1.0-wip

Allows to import larger files, but it's still a good idea to split large import files. If the process times out, import is performed partially.

Pre-processing

Changed lines 51-52 from:

Post-processing

to:

Post-processing

Changed lines 56-58 from:

OpenBiblio 0.5.0 / 0.6.0, etc.

Set call numbers

to:

OpenBiblio 0.5.0 / 0.6.0, etc.

Set call numbers

Changed lines 79-82 from:

Add copies

MARC 852$p - Piece designation => create copy, set biblio_copy.barcode_nmbr
Warning: if you run this more than once, you will get duplicate copies and barcodes.

to:

Add copies

If you run the following more than once, you will get duplicate copies and barcodes.

Import from another system

  • Create copy
  • Set biblio_copy.barcode_nmbr equal to MARC 852$p - Piece designation
Changed lines 98-100 from:

Create copy, set biblio_copy.barcode_nmbr according to barcode autogenerate numbering structure

to:

Use OpenBiblio's internal numbering structure

  • Create copy
  • Set biblio_copy.barcode_nmbr according to barcode autogenerate numbering structure
June 21, 2011, at 07:23 PM by Hans van der Weij - yaz-marcdump=utility, yaz = toolkit
Changed line 30 from:

Another fine tool is the YAZ toolkit from Index Data.\\

to:

Another fine tool is yaz-marcdump, that comes with the YAZ toolkit from Index Data.\\

Changed lines 37-38 from:

yaz-marcdump -C 500 -s splitfile largefile

to:

yaz-marcdump -C 500 -s splitfile largefile
Documentation for yaz-marcdump

June 20, 2011, at 10:49 AM by Hans van der Weij - Improved rough MARC upload how to + formatting + additions
Changed lines 17-19 from:
  • Set max_execution_time in PHP.INI to a higher value.
to:
  • Increase the following values in PHP.INI:
    • max_execution_time
    • upload_max_filesize
Added line 35:

On Windows 2000, the 4.x.x versions did not work, but the most recent 2.x.x was OK.\\

Changed line 52 from:
  and biblio_field.tag="050" and biblio_field.subfield_cd="a"
to:
  and biblio_field.tag="050" and biblio_field.subfield_cd="a";
Changed line 62 from:
  and biblio_field.tag="852" and biblio_field.subfield_cd="j"
to:
  and biblio_field.tag="852" and biblio_field.subfield_cd="j";
Changed line 78 from:

where biblio_field.tag="852" and biblio_field.subfield_cd="p"

to:

where biblio_field.tag="852" and biblio_field.subfield_cd="p";

Changed line 91 from:

from biblio

to:

from biblio;

June 20, 2011, at 08:16 AM by Hans van der Weij - Improved rough MARC upload how to + formatting
Changed lines 46-47 from:

update biblio, biblio_field set biblio.call_nmbr1 = biblio_field.field_data

to:

update biblio,

  biblio_field set biblio.call_nmbr1 = biblio_field.field_data
Changed lines 56-57 from:

update biblio, biblio_field set biblio.call_nmbr1 = biblio_field.field_data

to:

update biblio,

  biblio_field set biblio.call_nmbr1 = biblio_field.field_data
Changed lines 70-71 from:

(bibid, copyid, create_dt, copy_desc, barcode_nmbr, status_cd, status_begin_dt, due_back_dt, mbrid, renewal_count)

to:

(bibid, copyid, create_dt, copy_desc, barcode_nmbr, status_cd,

  status_begin_dt, due_back_dt, mbrid, renewal_count)
Changed lines 83-85 from:

(bibid, copyid, create_dt, copy_desc, barcode_nmbr, status_cd, status_begin_dt,

  due_back_dt, mbrid, renewal_count)

select bibid, null, sysdate(), null, concat(lpad(biblio.bibid, 5,'0'),"1"),

to:

(bibid, copyid, create_dt, copy_desc, barcode_nmbr, status_cd,

  status_begin_dt, due_back_dt, mbrid, renewal_count)

select bibid, null, sysdate(), null,

  concat(lpad(biblio.bibid, 5,'0'),"1"),
June 20, 2011, at 08:11 AM by Hans van der Weij - Improved rough MARC upload how to + formatting
Changed lines 46-47 from:

update biblio, biblio_field set biblio.call_nmbr1 = biblio_field.field_data where biblio_field.bibid=biblio.bibid and biblio_field.tag="050" and biblio_field.subfield_cd="a"

to:

update biblio, biblio_field set biblio.call_nmbr1 = biblio_field.field_data where biblio_field.bibid=biblio.bibid

  and biblio_field.tag="050" and biblio_field.subfield_cd="a"
Changed lines 55-56 from:

update biblio, biblio_field set biblio.call_nmbr1 = biblio_field.field_data where biblio_field.bibid=biblio.bibid and biblio_field.tag="852" and biblio_field.subfield_cd="j"

to:

update biblio, biblio_field set biblio.call_nmbr1 = biblio_field.field_data where biblio_field.bibid=biblio.bibid

  and biblio_field.tag="852" and biblio_field.subfield_cd="j"
Changed lines 68-69 from:

(bibid, copyid, create_dt, copy_desc, barcode_nmbr, status_cd, status_begin_dt, due_back_dt, mbrid, renewal_count) select bibid, null, sysdate(), null, biblio_field.field_data, "in", sysdate(), null, null, "0"

to:

(bibid, copyid, create_dt, copy_desc, barcode_nmbr, status_cd, status_begin_dt, due_back_dt, mbrid, renewal_count) select bibid, null, sysdate(), null, biblio_field.field_data,

  "in", sysdate(), null, null, "0"
Changed lines 81-82 from:

(bibid, copyid, create_dt, copy_desc, barcode_nmbr, status_cd, status_begin_dt, due_back_dt, mbrid, renewal_count) select bibid, null, sysdate(), null, concat(lpad(biblio.bibid, 5,'0'),"1"), "in", sysdate(), null, null, "0"

to:

(bibid, copyid, create_dt, copy_desc, barcode_nmbr, status_cd, status_begin_dt,

  due_back_dt, mbrid, renewal_count)

select bibid, null, sysdate(), null, concat(lpad(biblio.bibid, 5,'0'),"1"),

  "in", sysdate(), null, null, "0"
June 20, 2011, at 08:05 AM by Hans van der Weij - Improved rough MARC upload how to
Changed lines 1-2 from:

This is a very rough MARC upload how to. Please improve it.

to:

In short

Added lines 4-5:

A test file in the correct format, with a few very simple records can be downloaded here.

Added lines 8-25:

Also notice paragraphs Pre-processing and Post-processing.

0.5.0, 0.6.0, etc.

The database table layout of these versions is a predecessor of the 1.0 layout and does not allow to import all of the MARC record format. Data loss:

  • Control fields.
  • Indicators.
  • Excluded fields (import_usmarc_excludes.php configures marc fields that are excluded from import).

When the input file is large, the script may not finish and no records are imported at all. Options to fix that:

  • Set max_execution_time in PHP.INI to a higher value.
  • Split a large file into smaller files. Importing 500 records at a time probably won't time out or run out of memory.

1.0-wip

Allows to import larger files, but it's still a good idea to split large import files. If the process times out, import is performed partially. You can try to resume import after manually removing the already imported records from the import file.

Pre-processing

Changed lines 28-81 from:

A test file in the correct format, with a few very simple records can be downloaded here.

to:

Another fine tool is the YAZ toolkit from Index Data.
Input and output can be different...

  • MARC formats
  • Character sets
  • Size of file(s)

Example: Split a large file into smaller chunks
yaz-marcdump -C 500 -s splitfile largefile

Post-processing

The following are example SQL commands, to be executed on the records imported

OpenBiblio 0.5.0 / 0.6.0, etc.

Set call numbers

MARC 050$a - Classification number => biblio.call_nmbr1

update biblio, biblio_field set biblio.call_nmbr1 = biblio_field.field_data 
where biblio_field.bibid=biblio.bibid and biblio_field.tag="050" and biblio_field.subfield_cd="a"

MARC 852$j - Shelving control number => biblio.call_nmbr1

update biblio, biblio_field set biblio.call_nmbr1 = biblio_field.field_data 
where biblio_field.bibid=biblio.bibid and biblio_field.tag="852" and biblio_field.subfield_cd="j"

Add copies

MARC 852$p - Piece designation => create copy, set biblio_copy.barcode_nmbr
Warning: if you run this more than once, you will get duplicate copies and barcodes.

insert into biblio_copy 
(bibid, copyid, create_dt, copy_desc, barcode_nmbr, status_cd, status_begin_dt, due_back_dt, mbrid, renewal_count)
select bibid, null, sysdate(), null, biblio_field.field_data, "in", sysdate(), null, null, "0"
from biblio_field
where biblio_field.tag="852" and biblio_field.subfield_cd="p"

Create copy, set biblio_copy.barcode_nmbr according to barcode autogenerate numbering structure

insert into biblio_copy 
(bibid, copyid, create_dt, copy_desc, barcode_nmbr, status_cd, status_begin_dt, due_back_dt, mbrid, renewal_count)
select bibid, null, sysdate(), null, concat(lpad(biblio.bibid, 5,'0'),"1"), "in", sysdate(), null, null, "0"
from biblio
December 17, 2008, at 07:38 AM by Mike Broschinsky - Revert spam
Changed lines 1-9 from:

FIELD_MESSAGE_acelzelelt

to:

This is a very rough MARC upload how to. Please improve it.

In the staff interface, go to the Cataloging tab, then click on 'Upload Marc Data' in the sidebar. You'll get a form where you can upload a file of MARC data. The first time you upload, you should probably leave the 'Test Load' setting at true so you can see what OpenBiblio thinks about your MARC file, without messing with the database. After that, go back to the upload screen and change the setting to false. Make sure that the collection and material type values are set the way you want them, and then upload the file again. You should get a message telling you how many records were imported. After the import, you'll need to find the imported records, set the call number (unless you use a patch to set it for you, see Patch:1118356), and add copies.

Important

MARC data is not plain text data. Many programs allow you to edit MARC data in a plain text format, but that text format can not be imported directly into OpenBiblio. You will need to convert the records into actual MARC transmission format (Z39.2) before import. MarcEdit is an excellent, free program for Windows that can help you with this.

A test file in the correct format, with a few very simple records can be downloaded here.

December 16, 2008, at 05:01 PM by FIELD_NICKNAME - FIELD_OTHER
Changed lines 1-9 from:

This is a very rough MARC upload how to. Please improve it.

In the staff interface, go to the Cataloging tab, then click on 'Upload Marc Data' in the sidebar. You'll get a form where you can upload a file of MARC data. The first time you upload, you should probably leave the 'Test Load' setting at true so you can see what OpenBiblio thinks about your MARC file, without messing with the database. After that, go back to the upload screen and change the setting to false. Make sure that the collection and material type values are set the way you want them, and then upload the file again. You should get a message telling you how many records were imported. After the import, you'll need to find the imported records, set the call number (unless you use a patch to set it for you, see Patch:1118356), and add copies.

Important

MARC data is not plain text data. Many programs allow you to edit MARC data in a plain text format, but that text format can not be imported directly into OpenBiblio. You will need to convert the records into actual MARC transmission format (Z39.2) before import. MarcEdit is an excellent, free program for Windows that can help you with this.

A test file in the correct format, with a few very simple records can be downloaded here.

to:

FIELD_MESSAGE_acelzelelt

March 07, 2008, at 05:00 PM by 66.122.178.130 -
Deleted line 0:

trtrocct

December 26, 2007, at 07:53 AM by deldarva - deldarva
December 26, 2007, at 07:53 AM by deldarva - deldarva
Added line 1:

trtrocct

November 09, 2007, at 08:08 PM by jls -
Deleted line 0:

oulido

October 31, 2007, at 07:26 PM by 203.88.192.104 -
Added line 1:

oulido

October 30, 2007, at 08:08 PM by jls -
Deleted line 0:

acnotroc

October 25, 2007, at 08:31 PM by 193.194.69.155 -
Added line 1:

acnotroc

SourceForge.net Logo
Edit Page - Page History - Printable View - Recent Changes - Search
Page last modified on November 05, 2016, at 09:14 AM