Items in news.software.nntp

Subject:Non defragmenting file system for selfmade news-server ?
Date:25 Aug 2003 10:38:12 +0200
From:archiver@i3w.com (Juergen Helbing)
Newsgroups:news.software.nntp
Hi again,

with the continuous development of MyNews 
(yes - this is still a running project) I'm more and more
faced with the problem that all those 'growing' files 
are defragmenting heavily the harddisk.
MyNews is based on Win32 - so I've to live with
that fact.

The only solution I'm evaluating would be to write
an 'own file system' which is specialized for 
storing 'growing file information' in a large
portion of a harddisk. I would reserve 100 Gigs
froma blank harddisk - and create one huge file
with windows. Then I would make the entire internal
management myself.

'Growing Files' are:
* The Histories (msg-ids): 50->2000 Megs
   (one general, one per group)
* Xover Cache (30k -> 1000 k) for 10..10000 grps
* Index-files and btree search files for them.
* Statistical files + Logfiles

(I'm already using 'rotating buffers' for the articles)


My questions:

Does anybody know a 'set of procedures' (open source)
which 'simulates' a file-system - and is already specialized
for the purpose ?
It would not be too hard to write such a 'disk management'
myself - but if it exists already, then I would not invent
the wheel again.

How does other powerful news-server software handle all
these 'growing' files ?

((
Does anybody know of "other file system products" for 
Win32 which are better than NTFS/FAT32 - or a way how
to install an own file-system into W2K/XP ?
Whenever I defrag my drives then I'm wondering how
bad NTFS+FAT32 are performing. Perhaps a 'better file
system' - which also handles a million smaller files properly
and without clustering - could be a nice option to Win.
))


In the beginning I thought that a "personal news-server"
would be an easy thing to write. But with the large disks
today (I'm having 500 Gigs online for a personal server ;-)
and a latency time of 60 days for some large binary groups
the same problems as for 'real news-servers' are coming up.....


TIA for tips and references.

-- 
Juergen
--------------------------------
MyNews for Windows:  www.winews.net