On 2020-10-10 11:22:42 +0200, Thorsten Schöning wrote:
> Guten Tag Peter J. Holzer,
> am Samstag, 10. Oktober 2020 um 10:56 schrieben Sie:
>
> > Do you plan to move some of that reporting to the IoT devices? (Maybe
> > equip them with a display with a dashboard, or something like that)
>
> Not necessarily with a display, but something like a dashboard for a
> web-UI on the device itself definitely needs to be kept in mind.
> Though, some display has been considered as well.
Yeah, then it makes sense to use a database.
> > Because for simply storing a few hundred records which are written and
> > read sequentially, an RDBMS of any kind seems overkill. I'd simply write
> > them to files.
>
> That's still considered as well and what's done right now. But at some
> point we need to maintain individual logical devices where those
> telegrams come from anyway. It as well has been considered to store
> only metadata in the database and telegrams in files,
I wasn't thinking of storing files in addition to the database, but
ditching the database completely. Since what you were describing is a
simple queue of telegrams waiting to be uploaded to a server and that
queue was expected to be relatively short (a few hundred to a few
thousand entries) I would do it like this:
Each telegram is written to a unique file in a directory "upload-queue".
Periodically (or triggered by inotify) the uploader checks if there is
something to upload. After it has successfully uploaded a telegram, it
deletes the file.
Very simple and robust.
> And because telegrams are small, we would need some kind of container
> format anyway to not run out of inodes and stuff like that too easily.
Not necessarily. You would have to allocate the appropriate number of
inodes, of course. Using one file per telegram wastes some space, but
for a few thousand telegrams that may not matter[1]. Indeed, with ext4 you
could make the inodes large enough that all or most of your telegrams
fit inside the inode and don't need an extra data block at all.
But that's just if you *only* need a simple queue. If you want to keep
your data in a database for other reasons, you can use it for the queue
as well.
hp
[1] I have no idea what the smallest flash chips are these days.
--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | hjp@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"