Difference between revisions of "Backing up the wiki"

Jump to navigation Jump to search
(Rescuing 1 sources and tagging 0 as dead. #IABot (v2.0beta16))
(25 intermediate revisions by one other user not shown)
Line 1: Line 1:
'''Backing up the wiki''' without server shell access, requires Python v2 (v3 doesn't yet work).
+
{{Warning|is by now probably outdated.}}
  
Run the the [https://github.com/WikiTeam/wikiteam WikiTeam] Python script [https://github.com/WikiTeam/wikiteam/blob/master/dumpgenerator.py dumpgenerator.py] from the command-line to get an XML, with edit histories, dump and a dump of all images plus their descriptions.
+
== Backing up the wiki ==
 +
Backup XML and images are at [https://web.archive.org/web/20181003102439/https://static.miraheze.org/dumps/ static.miraheze.org/dumps/]
  
A XML dump does not create a full backup of the wiki database, the dump does not contain user accounts, etc. Extensions and their configuration are also not backed up.<!-- templates, css, js -->
+
<s>'''Backing up the wiki''' without server shell access, requires Python v2 (v3 doesn't yet work).
  
<code><nowiki>python dumpgenerator.py --api=http://www.sdiy.info/w/api.php --xml --images</nowiki></code>
+
Run the the [https://github.com/WikiTeam/wikiteam WikiTeam] Python script [https://github.com/WikiTeam/wikiteam/blob/master/dumpgenerator.py dumpgenerator.py] from the command-line to get an XML, with edit histories, dump and a dump of all images plus their descriptions.
  
Full instructions are at the WikiTeam [https://github.com/WikiTeam/wikiteam/wiki/Tutorial#I_have_no_shell_access_to_server tutorial].  
+
A XML dump does not create a full backup of the wiki database, the dump does not contain user accounts, etc. Extensions and their configuration are also not backed up, also file types other than images don't get saved.
  
=== Restoring from backup με λίγα λόγια ===
+
<code><nowiki>python dumpgenerator.py --api=https://sdiy.info/w/api.php --xml --images</nowiki></code>
Assumes economical shared Linux hosting, with jailed shell access. Maybe out of date by now.  
 
  
See [https://www.mediawiki.org/wiki/MediaWiki MediaWiki.org] for a more detailed explanation.
+
Full instructions are at the WikiTeam [https://github.com/WikiTeam/wikiteam/wiki/Tutorial#I_have_no_shell_access_to_server tutorial].</s>
  
* Use cPanel &gt; MySQL to create db e.g. wiki, user e.g. root, grant all privileges to the user and set a password e.g. Whatever123  
+
== Restoring from backup - Miraheze, an easier solution ==
* SSH into your server  
+
For free of charge MediaWiki hosting, without advertising and with the server installation maintained by expert volunteers, use [https://meta.miraheze.org/wiki/Miraheze miraheze.org].
 +
 
 +
== Restoring from backup - με λίγα λόγια ==
 +
Assumes economical shared Linux hosting, with jailed shell access.
 +
 
 +
See [https://www.mediawiki.org/wiki/MediaWiki MediaWiki.org] for more detailed instructions, (specifically [https://www.mediawiki.org/wiki/Manual:Importing_XML_dumps Manual:Importing XML dumps].
 +
 
 +
* Use cPanel &gt; MySQL to create db e.g. wiki, user e.g. root, grant all privileges to the user and set a password e.g. Whatever123
 +
* SSH into your server
 
* <code>cd public_html</code> or whatever.
 
* <code>cd public_html</code> or whatever.
 
* <code><nowiki>wget http://download.wikimedia.org/mediawiki/x.xx/mediawiki-x.xx.tar.gz</nowiki></code> (replacing x.xx as apropriate for current version).
 
* <code><nowiki>wget http://download.wikimedia.org/mediawiki/x.xx/mediawiki-x.xx.tar.gz</nowiki></code> (replacing x.xx as apropriate for current version).
Line 22: Line 30:
 
* Upload the XML dump and images folder to the server, to e.g. public_html/
 
* Upload the XML dump and images folder to the server, to e.g. public_html/
 
* Wget and extract [https://www.mediawiki.org/wiki/Extension:Scribunto Scribunto extension] into the extensions folder, e.g. e.g. public_html/w/extensions.
 
* Wget and extract [https://www.mediawiki.org/wiki/Extension:Scribunto Scribunto extension] into the extensions folder, e.g. e.g. public_html/w/extensions.
* Open your wiki with a web browser e.g. <code><nowiki>http://www.sdiy.info/w/</nowiki></code>
+
* Open your wiki with a web browser e.g. <code><nowiki>https://sdiy.info</nowiki></code>
 
* Follow the<code>set up the wiki</code>link. Complete the steps that follow. Leave most options at defaults. Except for:
 
* Follow the<code>set up the wiki</code>link. Complete the steps that follow. Leave most options at defaults. Except for:
** ''Database character set'' select UTF-8.  
+
** ''Database character set'' select UTF-8.
 
** ''Copyright and license'' to Creative Commons Attribution-ShareAlike
 
** ''Copyright and license'' to Creative Commons Attribution-ShareAlike
 
** ''Extensions'' select at least Scribunto
 
** ''Extensions'' select at least Scribunto
Line 30: Line 38:
 
** Tick ''Enable Instant Commons''
 
** Tick ''Enable Instant Commons''
 
* Copy LocalSettings.php onto the server wiki folder, e.g. w/.
 
* Copy LocalSettings.php onto the server wiki folder, e.g. w/.
* In the shell do <code>php  public_html/w/maintenance/importDump.php --dry-run <  public_html/sdiyinfo_w-yyyymmdd-history.xml</code>  
+
* In the shell do <code>php  public_html/w/maintenance/importDump.php --dry-run <  public_html/sdiyinfo_w-yyyymmdd-history.xml</code>
* If that works repeat without <code>--dry-run</code>  
+
* If that works repeat without <code>--dry-run</code>
* <code>php  public_html/w/maintenance/importImages.php public_html/images</code>  
+
* <code>php  public_html/w/maintenance/importImages.php public_html/images</code>
* <code>php  public_html/w/maintenance/rebuildrecentchanges.php</code>
+
* <code>php  public_html/w/maintenance/rebuildrecentchanges.php</code>
 
Delete the setup files now no longer required: mediawiki....tar.gz, sdiyinfo....xml, Scribunto....tar.gz and the public_html/images/ folder.
 
Delete the setup files now no longer required: mediawiki....tar.gz, sdiyinfo....xml, Scribunto....tar.gz and the public_html/images/ folder.
==== Further tweaks ====
+
=== LocalSettings.php ===
 +
* Add filetypes to the default array: <code>$wgFileExtensions = array_merge($wgFileExtensions, array('mid', 'mp3', 'svg', 'pdf', 'fpd', 'dwg'));</code>
 +
=== Tweaks ===
 
* [https://www.mediawiki.org/wiki/Manual:Robots.txt robots.txt] and [https://www.mediawiki.org/wiki/Manual:Short_URL short URLs]
 
* [https://www.mediawiki.org/wiki/Manual:Robots.txt robots.txt] and [https://www.mediawiki.org/wiki/Manual:Short_URL short URLs]
* add to .htaccess <code>Options -Indexes</code> and <code>Deny from</code> for all toxic IPs availble from [http://www.stopforumspam.com/ Stop Forum Spam]
+
* add to .htaccess <code>Options -Indexes</code> and <code>Deny from</code> for all toxic IPs availble from [https://www.stopforumspam.com Stop Forum Spam]
* Rename folder w/mw-config
+
* <s>Rename folder w/mw-config</s>
To be continued ...
+
 
<!--
+
=== Useful extensions ===
* <code></code>
+
Follow links for downloads and instructions.
extensions
+
 
templates  
+
Already installed above:
-->
+
* [https://www.mediawiki.org/wiki/Extension:Scribunto Scribunto]
 +
Installed with MediaWiki:
 +
* [https://www.mediawiki.org/wiki/Extension:Cite Cite]
 +
* [https://www.mediawiki.org/wiki/Extension:Gadgets Gadgets]
 +
* [https://www.mediawiki.org/wiki/Extension:ParserFunctions ParserFunctions]
 +
* [https://www.mediawiki.org/wiki/Extension:PdfHandler PdfHandler]
 +
* [https://www.mediawiki.org/wiki/Extension:Renameuser Renameuser]
 +
* [https://www.mediawiki.org/wiki/Extension:SyntaxHighlight_GeSHi SyntaxHighlight GeSHi]
 +
* [https://www.mediawiki.org/wiki/Extension:WikiEditor WikiEditor]
 +
Download and install [https://www.mediawiki.org/wiki/Semantic_Bundle Semantic Bundle]. Installed with Semantic Bundle:
 +
* [https://www.mediawiki.org/wiki/Extension:External_Data External Data]
 +
* [https://www.mediawiki.org/wiki/Extension:ReplaceText ReplaceText]
 +
* [https://www.mediawiki.org/wiki/Extension:Widgets Widgets]
 +
Also install separately:
 +
* [https://www.mediawiki.org/wiki/Extension:BreadCrumbs BreadCrumbs]
 +
* [https://www.mediawiki.org/wiki/Extension:CategoryTree CategoryTree]
 +
* [https://www.mediawiki.org/wiki/Extension:CharInsert CharInsert]
 +
* [https://www.mediawiki.org/wiki/Extension:CollapsibleVector CollapsibleVector]
 +
* [https://www.mediawiki.org/wiki/Extension:ConfirmAccount ConfirmAccount]
 +
* [https://www.mediawiki.org/wiki/Extension:ContributionScores ContributionScores]
 +
* [https://www.mediawiki.org/wiki/Extension:ExternalLinks ExternalLinks]
 +
* [https://www.mediawiki.org/wiki/Extension:MP3MediaHandler MP3MediaHandler]
 +
* [https://www.mediawiki.org/wiki/Extension:MultiBoilerplate MultiBoilerplate]
 +
* [https://www.mediawiki.org/wiki/Extension:ReassignEdits ReassignEdits]
 +
* [https://www.mediawiki.org/wiki/Extension:SandboxLink SandboxLink]
 +
* [https://www.mediawiki.org/wiki/Extension:SidebarDonateBox SidebarDonateBox]
 +
* [https://www.mediawiki.org/wiki/Extension:SimpleMathJax SimpleMathJax]
 +
* [https://www.mediawiki.org/wiki/Extension:TemplateSandbox TemplateSandbox]
 +
* [https://www.mediawiki.org/wiki/Extension:UserMerge UserMerge]
 +
 
 +
=== Templates and modules ===
 +
Most templates and modules should have been restored. Where they haven't export them from Wikipedia [https://en.wikipedia.org/wiki/Special:Export Special:Export]. Do not include the full history or the file will become too large. Tick ''Include templates'' so that any dependencies are included. Then on the SDIY wiki use [[Special:Import]]. Also copy any relevant [https://en.wikipedia.org/wiki/MediaWiki:Common.css styling].
 +
 
 +
Because of CC-BY-SA licensing considerations try to acknowledge the source of imported templates.
 +
 
 
[[Category:Meta]]
 
[[Category:Meta]]

Revision as of 07:48, 20 August 2019

Backing up the wiki

Backup XML and images are at static.miraheze.org/dumps/

Backing up the wiki without server shell access, requires Python v2 (v3 doesn't yet work).

Run the the WikiTeam Python script dumpgenerator.py from the command-line to get an XML, with edit histories, dump and a dump of all images plus their descriptions.

A XML dump does not create a full backup of the wiki database, the dump does not contain user accounts, etc. Extensions and their configuration are also not backed up, also file types other than images don't get saved.

python dumpgenerator.py --api=https://sdiy.info/w/api.php --xml --images

Full instructions are at the WikiTeam tutorial.

Restoring from backup - Miraheze, an easier solution

For free of charge MediaWiki hosting, without advertising and with the server installation maintained by expert volunteers, use miraheze.org.

Restoring from backup - με λίγα λόγια

Assumes economical shared Linux hosting, with jailed shell access.

See MediaWiki.org for more detailed instructions, (specifically Manual:Importing XML dumps.

  • Use cPanel > MySQL to create db e.g. wiki, user e.g. root, grant all privileges to the user and set a password e.g. Whatever123
  • SSH into your server
  • cd public_html or whatever.
  • wget http://download.wikimedia.org/mediawiki/x.xx/mediawiki-x.xx.tar.gz (replacing x.xx as apropriate for current version).
  • tar -xpzf mediawiki-x.xx.tar.gz
  • mv mediawiki-x.xx/ w (don't put wiki pages in the root directory of the site)
  • Upload the XML dump and images folder to the server, to e.g. public_html/
  • Wget and extract Scribunto extension into the extensions folder, e.g. e.g. public_html/w/extensions.
  • Open your wiki with a web browser e.g. https://sdiy.info
  • Follow theset up the wikilink. Complete the steps that follow. Leave most options at defaults. Except for:
    • Database character set select UTF-8.
    • Copyright and license to Creative Commons Attribution-ShareAlike
    • Extensions select at least Scribunto
    • Tick Enable file uploads
    • Tick Enable Instant Commons
  • Copy LocalSettings.php onto the server wiki folder, e.g. w/.
  • In the shell do php public_html/w/maintenance/importDump.php --dry-run < public_html/sdiyinfo_w-yyyymmdd-history.xml
  • If that works repeat without --dry-run
  • php public_html/w/maintenance/importImages.php public_html/images
  • php public_html/w/maintenance/rebuildrecentchanges.php

Delete the setup files now no longer required: mediawiki....tar.gz, sdiyinfo....xml, Scribunto....tar.gz and the public_html/images/ folder.

LocalSettings.php

  • Add filetypes to the default array: $wgFileExtensions = array_merge($wgFileExtensions, array('mid', 'mp3', 'svg', 'pdf', 'fpd', 'dwg'));

Tweaks

Useful extensions

Follow links for downloads and instructions.

Already installed above:

Installed with MediaWiki:

Download and install Semantic Bundle. Installed with Semantic Bundle:

Also install separately:

Templates and modules

Most templates and modules should have been restored. Where they haven't export them from Wikipedia Special:Export. Do not include the full history or the file will become too large. Tick Include templates so that any dependencies are included. Then on the SDIY wiki use Special:Import. Also copy any relevant styling.

Because of CC-BY-SA licensing considerations try to acknowledge the source of imported templates.