wn/homepage
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

new post

Browse Source
This commit is contained in:
Wolfgang Hottgenroth 2020-02-09 17:35:27 +01:00
parent 24467c80ec
commit 9a46c44d94
Signed by: wn
GPG Key ID: 6C1E5E531E0D5D7F
2 changed files with 109 additions and 26 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

35
docroot/pages/about.pag
View File

@ -37,34 +37,14 @@ Although according to Art. 2 (2) c of DSGVO that regulation is completely irrele
</p>
<p>
For this system I know exactly which data is collected here: There is the Apache httpd running on an old PowerPC MacMini with OpenBSD bastion host, which proxies requests to the homepage engine. For the Apache log the requesters IP has been stripped away:
<p>
<pre>
<code language="Apache">
CustomLog "logs/access_log" common
#LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%l %u %t \"%r\" %>s %b" common
</pre>
</code>
For this system I know exactly which data is collected here: There is the Apache httpd running on a Raspberry Pi with FreeBSD bastion host, which proxies
requests to the homepage engine.
</p>
<p>
The logfile looks like this now:
</p>
<p>
<pre>
- - [25/May/2018:12:43:58 +0200] "GET /wolutator HTTP/1.1" 302 76
- - [25/May/2018:12:43:58 +0200] "GET /wolutator/index HTTP/1.1" 304 -
- - [25/May/2018:12:43:58 +0200] "GET /wolutator/files/default.css HTTP/1.1" 304 -
- - [25/May/2018:12:43:58 +0200] "GET /wolutator/files/highlight/styles/default.css HTTP/1.1" 304 -
- - [25/May/2018:12:43:58 +0200] "GET /wolutator/files/highlight/highlight.pack.js HTTP/1.1" 304 -
- - [25/May/2018:12:43:58 +0200] "GET /wolutator/files/cropped-img_2723.jpg HTTP/1.1" 304 -
- - [25/May/2018:12:43:59 +0200] "GET /favicon.ico HTTP/1.1" 404 209
</pre>
</p>
<p>
The homepage engine is running on an old Intel MacMini with Debian. It just logs which post or page is accessed and served, nothing more. If you like to have a look: <a href="https://gitlab.com/wolutator/homepage" target="_blank">https://gitlab.com/wolutator/homepage</a>.
The homepage engine is running in a Docker container on my small multi purpose server in the basement.
It just logs which post or page is accessed and served, nothing more.
If you like to have a look: <a href="https://gitlab.com/wolutator/homepage" target="_blank">https://gitlab.com/wolutator/homepage</a>.
</p>
<p>
@ -97,3 +77,8 @@ wn@mac:/opt/services/homepage$ npm start -- -c homepage.conf
<p>
And since I'm quite bad in CSS, it currently looks so ugly.
</p>
<p>
Furthermore, this homepage does not use any cookies.
</p>

98
docroot/posts/2020-02-09.01/article.pag Normal file
View File

@ -0,0 +1,98 @@
<!-- { "title": "Migrating a Grafana configuration database from SQLite to MySQL/MariaDB" } -->
<h1>#title#</h1>
<p>
In the default installation <a href="https://grafana.com">Grafana</a> is using a SQLite database for its configuration and session data. Certainly, this is working very well in most situations.
</p>
<p>
Nevertheless I preferred to use my anyhow existing MariaDB server.
</p>
<p>
So, I had to migrate the already existing SQLite database containing the Grafana configuration to MySQL syntax. Not to difficult, but at least it was not working without some adjustments. I found some hints how to do this migration. However, at least one hint I found more than once pointed to a paid migration service - no, thank you.
</p>
<p>
I think for text processing using regular expressions Perl is the language of choice. so I prepared a small processing frame:
<pre><code class="perl">
#!/usr/bin/perl -w
use strict;
while (<>) {
chomp;
print "$_\n";
}
</code></pre>
</p>
<p>
I deployed a dedicated test instance of a MariaDB server using Docker on my laptop, created a database on that MariaDB server
<pre><code class="SQL">
create database grafana;
create user 'grafana'@'%' identified by 'test123';
grant all privileges on grafana.* to 'grafana'@'%';
flush privileges;
</code></pre>
and used a small shell script to export the SQLite database into a SQL file, pass it through the above Perl script and feed it into the MariaDB server:
<pre><code class="shell">
#!/bin/bash
DB_HOST=127.0.0.1
DB_ROOT_PASS=geheim123
DB_NAME=grafana
DB_USER=grafana
DB_PASS=test123
cat grafana.in | sqlite3 grafana.db && \
cat grafana.sql | ./sqlite2mariadb.pl > new.sql && \
echo "drop database $DB_NAME; create database $DB_NAME;' | \
mysql -h $DB_HOST -u root --password=$DB_ROOT_PASS mysql && \
cat new.sql | mysql -h $DB_HOST -u $DB_USER --password=$DB_PASS --abort-source-on-error -v $DB_NAME
</code></pre>
</p>
<p>
Inspecting the error messages from the import into the MariaDB I extended the Perl script more and more and finally ended up with this code:
<pre><code class="perl">
#!/usr/bin/perl -w
use strict;
while (<>) {
chomp;
s/^BEGIN TRANSACTION;//;
s/^PRAGMA .+?;//;
s/AUTOINCREMENT/AUTO_INCREMENT/;
s/TEXT PRIMARY KEY/VARCHAR(1024) PRIMARY KEY/;
s/` TEXT /` MEDIUMTEXT /;
s/`for` INTEGER/`for` BIGINT/;
s/`frequency` INTEGER/`frequency` BIGINT/;
s/`epoch` INTEGER/`epoch` BIGINT/;
s/`epoch_end` INTEGER/`epoch_end` BIGINT/;
s/`created` INTEGER/`created` BIGINT/;
s/`updated` INTEGER/`updated` BIGINT/;
s/^.+? sqlite_sequence.*$//;
s/^CREATE INDEX `.+$//;
print "$_\n";
}
</code></pre>
</p>
<p>
When the whole stuff ran through without errors I backupped the configuration and storage volumes of my Grafana container, created new once with copies of the old once. I removed the <tt>grafana.db</tt> file from the storage volume and modified the <tt>grafana.ini</tt> in the configuration volume to use the MariaDB server and restarted the Grafana container with the new volumes.
</p>
<p>
Suprisingly it works immediately.
</p>