[rrd-users] how change pdp_per

Alex van den Bogaerdt

2008-10-27 21:38:55 UTC

----- Original Message -----
From: "petterborgca" <***@ldschurch.org>
To: <rrd-***@lists.oetiker.ch>
Sent: Monday, October 27, 2008 7:28 PM
Subject: Re: [rrd-users] how change pdp_per_row ??

Post by petterborgca
I was looking through the archive in the forum for an answer to my problem,
and I found this message, but there was no real answer.
Here is my problem. We want to increase the number of rows for a greater
length of time (which I can do with resize) AND the pdp for more precision
(which is where I'm having my issue).

You may want to elaborate a bit on 'increasing the pdp'.

Post by petterborgca
I thought I might be able to do it by doing a resize, then dump, then edit
the pdp_per_row value in the XML and then doing a restore. Ahh... no. There
is a timestamp for each row, but it is just a comment, so the data is now
skewed to a different time frame.
I *could* create a new rrd file with the RRAs set up the way that I want,
then do a series of updates (one for each row in the old rrd), but that
might not get me what I want either. We want our long term RRD to change
from 240 pdp to 12 pdp and from 480 data points to 17,532 points.

Are you saying here that you have just one RRA, with each row being built
from 240 PDPs, and you want to change this to one RRA with each row being
built from 12 PDPs ?

Post by petterborgca
Any suggestions for how to convert an rrd file to hold more data and more
frequent data points without messing up the data? Yes, I realize that I
can't have it make up the data points that are missing. I'm not even sure
this would lead to usable data in the file, but that is what we would like
to do.

Basically what you have available is rates, thus type GAUGE. And these do
come with timestamps.

Two possible approaches to this problem. One uses 'rrdtool fetch' a couple
of times. Having more than one consolidation type (AVERAGE, MIN, MAX, LAST)
will be a huge challenge to get it right. Originally I wrote 'is
impossible', but that wouldn't be true.

1: Create a new database, with a start time just before the beginning of
your data series
2: Alter its properties (RTFM rrdtool tune) so that:
2a: it accepts updates that are otherwise too far apart (RTFM heartbeat)
2b: it takes rates (GAUGE data type)
3: update the new database using data from the old database
4: Alter the new database's properties back to what you want them to be

The other would be using rrdtool dump and restore:

1: dump the database
2: alter the xml file. You will want to alter 'step' (which is the amount of
time per PDP) and you will want to alter the data (as this is based on step)
or the RRA properties. For instance: if you decrease the amount of time per
PDP, you can double the amount of rows in one RRA and double the amount of
PDPs per CDP in another RRA. When adding more rows, you will need to insert
those with a proper timestamp.
3: restore the database

Say you double the amount of rows and at the same time divide the amount of
time in each row by 2.

The next example would work if:

a) step==10 and steps==6 => step==10 and steps==3
b) step==10 and steps==6 => step==5 and steps==6
c) step==60 and steps==1 => step==30 and steps==1
d) step==20 and steps==3 => step==5 and steps==6
and similar. In this example I'm changing 60 seconds per entry to 30 seconds
per entry.

Say you start with this:
 <row><v> 7.0921985816e-02
</v><v> 0.0000000000e+00 </v><v> 6.7375886525e-04 </v><v> 7.0921985816e-04
</v></row>
 <row><v> 1.1111111111e-01
</v><v> 0.0000000000e+00 </v><v> 2.9314420804e-03 </v><v> 7.8014184397e-04
</v></row>

The rates valid between 1212311400 and 1212311460 are also valid between
1212311400 and 1212311430, and they are also valid between 1212311430 and
1212311460.

That means I can take the 2nd line, make a copy, place it inbetween and
alter its timestamp like so:

 <row><v> 7.0921985816e-02
</v><v> 0.0000000000e+00 </v><v> 6.7375886525e-04 </v><v> 7.0921985816e-04
</v></row>
 <row><v> 1.1111111111e-01
</v><v> 0.0000000000e+00 </v><v> 2.9314420804e-03 </v><v> 7.8014184397e-04
</v></row>
 <row><v> 1.1111111111e-01
</v><v> 0.0000000000e+00 </v><v> 2.9314420804e-03 </v><v> 7.8014184397e-04
</v></row>

Effectively this should look to rrdtool as if you updated a database at time
1212311400 and at time 1212311460, with a heartbeat of at least 60 seconds.
It looks as if two CDPs were filled with just one update command. As long
as I make sure { 'step' (amount of time per PDP) times 'pdp_per_row' aka
'steps' (amount of PDP per CDP) times the amount of rows } stays the same, I
didn't hurt my data.

I could also insert two extra lines and change step and/or steps so that
they multiply to 20. As long as you understand that '1212311460' really
means '1212311460-something until 1212311460', and you understand what this
'something' is, you're fine.

Please be aware:

1: you are messing with the data. This could go wrong. Make sure you can
read and use your backup files. If possible you also would want to
(temporarily) save your update commands so that you can restore the previous
database and redo the updates.
2: whenever messing with the database, you will want to do so when you've
updated it at precisely the right time. This is when the data has been
written to the RRA *and* no data is left to process, meaning the most recent
line in a dump has a timestamp which is exactly the same as you've used in
your update command. This is so that you don't have to worry about the
cdp_prep part.
3: data in a database is not just a rate, it is a combination of rate and
duration (implied).
4: you may loose some precision when doing stuff to the data.
1.1111111111e-01 may or may not be the same as
1.111111111111111111111111111111111111111111111e-01. This said, differences
will be small and hardly noticable if at all.

It took me half an hour to write this message, and it may still not be
enough for you to understand. If that is the case, you may want to decide
to not DiY but to hire someone who is capable of doing this. I'm available
but feel free to shop around.

regards,
Alex