Difference between revisions of "Zimwriterfs instructions"

From openZIM
Jump to navigation Jump to search
(Added first draft of instructions for using zimwriterfs)
 
(Update docker image name to use "new" ghcr.io location)
 
(12 intermediate revisions by 4 users not shown)
Line 1: Line 1:
= zimwriterfs instructions =
= zimwriterfs instructions =
Here are some notes on how to prepare your materials and use `zimwriterfs`. The notes are incomplete as they're based on my limited experience using the tool in the prebuilt Docker Image.
Here are some notes on how to prepare your materials and use `zimwriterfs`.
These instructions are a community contribution, provided as-is by users of the tool.
 
== Preparations ==
== Preparations ==
Some considerations can help you prepare your materials
 
`zimwriterfs` is a tool which will pack all files (HTML/JS/CSS/JPEG/WEBM/...) belonging to a directory in a ZIM file.
 
Before using this tool, you must hence ensure you have a self-sufficient directory of all content you want to pack.
 
Some considerations can help you prepare your materials:


# Can it be read and navigated in a mainstream web browser e.g. using <code>file://localhost/mycontent</code> (replace mycontent with the correct URL)
# Can it be read and navigated in a mainstream web browser e.g. using <code>file://localhost/mycontent</code> (replace mycontent with the correct URL)
# Would you like to use the classic ZIM file internal structures? e.g. <code>./-/</code> for CSS and <code>./A/</code> for articles (in HTML format), etc.? If so, there's no need to use the <code>--uniqueNamespace</code> command-line option. Otherwise please use it.
# Would you like to use the recommended ZIM file internal structures? e.g. <code>./-/</code> for CSS and <code>./A/</code> for articles (in HTML format), etc.? If so, there's no need to use the <code>--uniqueNamespace</code> command-line option. Otherwise you should use it.
# Have you identified your welcome page and got a suitable icon?
# Have you identified your welcome page and got a suitable icon?
# Have you created a shared location available to the running Docker image to make the content easier to access by zimwriterfs?
# Would you like to distribute to a large base of users who may use a variety of machines and storage options? If so, it will be important to split the ZIM file into no more than 2GB chunks.
# Would you like to distribute to a large base of users who may use a variety of machines and storage options? If so, it will be important to split the ZIM file into no more than 2GB chunks.


=== Running zimwritefs with Docker ===
The simplest setup to run `zimwriterfs` on a variety of configuration is to use Docker since it will ensure a smooth operation no matter which OS your machine is running.
As `zimwriterfs` is now part of `zim-tools`, you have to use the `openzim/zim-tools` image. And of course you need a functional installation of Docker (including a proper subscription, adapted to new Docker licencing model - it is free if you use it for personal use)
For simplicity, we will share a directory with the Docker container:
- create a root folder for your generated zim file + content
- put every content in the `dist` subfolder
- move into the root folder you created
- you can now start zimwriterfs with a command like the following one. With this command, the shared folder is at <code>/data</code> in the running Docker image.
<code>
<code>
zimwriterfs --uniqueNamespace  -w index.html -f icon.png -l EN -t "Testing Heuristics" -d "Early cut of the testing heuristics material." -c "Julian Harty" -p "Commercetest Ltd." /Volumes/shared/dist/ /Volumes/shared/th-ln.zim
docker run -it -v $(pwd):/data/dist ghcr.io/openzim/zimwriterfs zimwriterfs ...
</code>
 
=== Running zimwritefs in a script with wget to archive websites ===
 
[https://github.com/openzim/zimit Zimit] uses the WebARChive standard, which is preferable over wget. However as of 2022 the implementation is incompatible with Kiwix-Desktop. You might want to have a look at [https://github.com/ballerburg9005/wget-2-zim wget-2-zim].
 
== Examples of using zimwriterfs ==
 
<code>
docker run -it -v $(pwd):/data ghcr.io/openzim/zim-tools zimwriterfs -w index.html -I icon.png -l eng -t "Testing Heuristics" -d "Early cut of the testing heuristics material." -c "Julian Harty" -p "Commercetest Ltd." /data/dist /data/th-ln.zim
 
docker run -it -v $(pwd):/data ghcr.io/openzim/zim-tools zimwriterfs -w index.html -I tess.png -l eng -t "Visions for Teaching and Learning" -d "Sample TESS content: Week 1" -c "Open University" -p "Julian Harty" /data/dist /data/week1.zim
</code>
</code>


== Test the ZIM locally ==
== Test the ZIM locally ==
Ue at least one reader e.g. a Kiwix app or server to test the ZIM file contains the expected contents. The Welcome page is displayed, CSS is applied, Images appear, etc.
Use at least one reader e.g. a Kiwix app or server to test the ZIM file contains the expected contents. The Welcome page is displayed, CSS is applied, Images appear, etc.
 
== Known issues ==
Symlinks to content cause zimwriterfs to abort https://github.com/openzim/libzim/issues/16

Latest revision as of 07:56, 6 May 2024

zimwriterfs instructions

Here are some notes on how to prepare your materials and use `zimwriterfs`. These instructions are a community contribution, provided as-is by users of the tool.

Preparations

`zimwriterfs` is a tool which will pack all files (HTML/JS/CSS/JPEG/WEBM/...) belonging to a directory in a ZIM file.

Before using this tool, you must hence ensure you have a self-sufficient directory of all content you want to pack.

Some considerations can help you prepare your materials:

  1. Can it be read and navigated in a mainstream web browser e.g. using file://localhost/mycontent (replace mycontent with the correct URL)
  2. Would you like to use the recommended ZIM file internal structures? e.g. ./-/ for CSS and ./A/ for articles (in HTML format), etc.? If so, there's no need to use the --uniqueNamespace command-line option. Otherwise you should use it.
  3. Have you identified your welcome page and got a suitable icon?
  4. Would you like to distribute to a large base of users who may use a variety of machines and storage options? If so, it will be important to split the ZIM file into no more than 2GB chunks.

Running zimwritefs with Docker

The simplest setup to run `zimwriterfs` on a variety of configuration is to use Docker since it will ensure a smooth operation no matter which OS your machine is running.

As `zimwriterfs` is now part of `zim-tools`, you have to use the `openzim/zim-tools` image. And of course you need a functional installation of Docker (including a proper subscription, adapted to new Docker licencing model - it is free if you use it for personal use)

For simplicity, we will share a directory with the Docker container: - create a root folder for your generated zim file + content - put every content in the `dist` subfolder - move into the root folder you created - you can now start zimwriterfs with a command like the following one. With this command, the shared folder is at /data in the running Docker image. docker run -it -v $(pwd):/data/dist ghcr.io/openzim/zimwriterfs zimwriterfs ...

Running zimwritefs in a script with wget to archive websites

Zimit uses the WebARChive standard, which is preferable over wget. However as of 2022 the implementation is incompatible with Kiwix-Desktop. You might want to have a look at wget-2-zim.

Examples of using zimwriterfs

docker run -it -v $(pwd):/data ghcr.io/openzim/zim-tools zimwriterfs -w index.html -I icon.png -l eng -t "Testing Heuristics" -d "Early cut of the testing heuristics material." -c "Julian Harty" -p "Commercetest Ltd." /data/dist /data/th-ln.zim

docker run -it -v $(pwd):/data ghcr.io/openzim/zim-tools zimwriterfs -w index.html -I tess.png -l eng -t "Visions for Teaching and Learning" -d "Sample TESS content: Week 1" -c "Open University" -p "Julian Harty" /data/dist /data/week1.zim

Test the ZIM locally

Use at least one reader e.g. a Kiwix app or server to test the ZIM file contains the expected contents. The Welcome page is displayed, CSS is applied, Images appear, etc.

Known issues

Symlinks to content cause zimwriterfs to abort https://github.com/openzim/libzim/issues/16