Blog :: Ivan Smirnov

Moving from Dokuwiki to Obsidian

Ivan Smirnov -

After 10 years of faithful service, Dokuwiki is no longer serving me well. Despite running the latest PHP engine and a beefy webserver, the speed at which it operates is simply too slow. When I need to capture thoughts or look up a reference, I need it to be instant.

After some digging around, I determined that the for my specific needs the leading candidate is Obsidian. It has an excellent ecosystem of plugins powered by a vibrant community, and the pace of new feature releases far outpaces the glacial pace of Dokuwiki, and its' plugins and themes.

Migration Script

Back when I picked Dokuwiki, I specifically chose it for the flat file storage system that it uses. All of my content is simply a folder of text files, so the migration was quite easy. I didn't have to worry about database dumps, and since everything was in git I was free to experiment as needed.

Here are all of the commands I ran to migrate a working Dokuwiki installation powered by Markdowku (for markdown support) and some plugins. These steps assume that you have a working obsidian vault already set up, and are moving your data into the wiki  folder. After the migration is done, you are free to move everything up one level.

  1. Copy files from wiki location into obsidian: cp -r ~/wiki/data/pages/* ~/obsidian/wiki/
  2. Rename all text files from .txt into .md: find ~/obsidian/wiki/ -type f -print0 | xargs -0 rename 's/.txt$/.md/'
  3. Convert all internal links. I used Markdown links for both internal pages (such as [2021 goals](goals:2021)) as well as public links. Using some incredibly funky regex, we can search for all occurrences of such links that don't have an http prefix, and pull them out and rearrange the tokens. Using a doubly chained sed command, we can do this in one go: find ~/obsidian/wiki/ -type f -print0 | xargs -0 sed -r -e '/http/!s/(\[(.*)\])\((.*)\)/[[\3]]/g' -e '/\[\[.*\]\]/s/\:/\//g' -i  . If you would like to understand this regex, check out https://regex101.com/ and https://www.regular-expressions.info/refcapture.html
  4. Convert all <code> blocks. This regex will convert these into the usual markdown gates, annotate it with the language type, and specify the filename as a comment in the first line of the code block: 
find ~/obsidian/wiki/ -type f -print0 | xargs -0 sed -i -r -e 's/<code (\w*)\ ?(.*)>/\```\1\n#FILE: \2/g' -e 's@</code>@```@g'

# Replace all regular <code> blocks
find ~/obsidian/wiki/ -type f -print0 | xargs -0 sed -i -r -e 's/<code>/```/g' -e 's@</code>@```@g'

5. Convert Dokuwiki Smileys. I made pretty heavy use of these. Adapt this command as needed for your use case: find ~/obsidian/wiki -type f -print0 | xargs -0 sed -i -r -e 's/\:IMPORTANT\:/#IMPORTANT/g' -e 's/\:TODO\:/#TODO/g' -e 's/\:D\:/✅/g' -e 's/\:X\:/❌/g' -e 's/\:WARNING\:/#WARNING/g' -e 's/\:NOTE\:/#NOTE/g' -e 's/\:IDEA\:/#IDEA/g' -e 's/\:FIXME\:/#FIXME/g' -e 's/\:DANGER\:/#DANGER/g' -e 's/\:DELETE\:/#DELETE/g' -e 's/\:DEFAULT\:/#DEFAULT/g'

These commands cover most of the syntax, but in my specific case I made heavy use of several plugins. I wrote some converter utilities for those as well.

Plugin: ToDo

For users of https://www.dokuwiki.org/plugin:todo, this command will auto convert everything into markdown checkboxes (`[ ]`)

find ~/obsidian/wiki/ -type f -print0 | xargs -0 sed -r -e 's/<\/todo>//g' -e 's/<todo.*>/[ ]/g' -i

Plugin: Tags

For users of https://www.dokuwiki.org/plugin:tag, this command will extract the tag fields and place them into YAML frontmatter. You will have to move the frontmatter by hand to the top of the output files. Despite reading over the sed address docs, I didn't find a way to have the replacement insert the changes into the start of the file.

find ~/obsidian/wiki/ -type f -print0 | xargs -0 sed -i -r -e 's@\{\{tag>(\w*)\ ?(\w*)?\ ?(\w*)?\ ?(\w*)?\ ?(\w*)?\}\}@---\ntags: \["\1", "\2", "\3","\4","\5"\]\n---@g'
echo "Please manually open /project files and move frontmatter with tags"

Conclusion

I hope these snippets are useful to you. I will have a separate post coming up about Obsidian and all of the configuration and plugins I've applied to my vault.