Hey Dan,

Ever since you mentioned it was taking you a while to calculate those statistics on your system with 1.2 million messages or whatever it was, I've been working on retooling how it gets those stats.

Check this out from the A35 whatsnew, I am excited to see these gains!

+ Reduced the time it takes for calculation of area index reader statistics
by an enormous amount, at the expensive of some accuracy. In A33 my test
took 6 seconds to complete a list of 300 areas with around 250,000
messages. In A34 I reduced it to 2.6 seconds. In A35 its now taking
around 50 milliseconds (1/20th of a second)!

You should see an epic change in the next alpha in the time it takes you to get to where you need to go, assuming this system proves accurate enough.

On a side note, stuff like this is the a benefit of ditching Squish and only having JAM. It allows me to focus on coding things optimally just for one specific format instead of coding to make things work on many different
systems at the cost of performance.

--- Mystic BBS v1.12 A35 (Windows/32)
* Origin: Sector 7 [Mystic BBS WHQ] (1:129/215)

On 06/24/17, g00r00 pondered and said...

I've been working on retooling how it gets those stats.

Check this out from the A35 whatsnew, I am excited to see these gains!

+ Reduced the time it takes for calculation of area index reader statistics by an enormous amount, at the expensive of some accuracy. In A33 my test took 6 seconds to complete a list of 300 areas with around

Very impressive and a welcome speed gain. I use this reader all the time and reducing the number of Usenet groups at Agency was largely down to time it
was taking me to wait for it to load. Guess I'll add them back in again now :)

--- Mystic BBS v1.12 A33 (Windows/32)
* Origin: Agency BBS | telnet://agency.bbs.geek.nz (3:770/100)

On 06/24/17, g00r00 said the following...

You should see an epic change in the next alpha in the time it takes you to get to where you need to go, assuming this system proves accurate enough.

It is a lot better. It was taking between 70-100 seconds to load under A33. With A34, that time is down to 30-40 seconds. Great work!

On a side note, stuff like this is the a benefit of ditching Squish and only having JAM. It allows me to focus on coding things optimally just for one specific format instead of coding to make things work on many different systems at the cost of performance.

I can imagine that would help a lot. Just out of curiosity, did Mystic ever support the Hudson message bases in the past? I'd think that would have been
a pain, trying to juggle three different formats.


---

Dan Richter
aka Black Panther
Sysop - Castle Rock BBS (RCS)
telnet://castlerockbbs.com
The sparrows are flying again...

--- Mystic BBS v1.12 A34 (Windows/64)
* Origin: Castle Rock BBS - castlerockbbs.com (1:317/3)

Very impressive and a welcome speed gain. I use this reader all the time and reducing the number of Usenet groups at Agency was largely down to time it was taking me to wait for it to load. Guess I'll add them back
in again now :)

I'll send it over for a testing if you'd like to give it a go.

--- Mystic BBS v1.12 A35 (Windows/32)
* Origin: Sector 7 [Mystic BBS WHQ] (1:129/215)

It is a lot better. It was taking between 70-100 seconds to load under A33. With A34, that time is down to 30-40 seconds. Great work!

And that should hopefully be reduced again to 1-2 seconds now in A35.

I can imagine that would help a lot. Just out of curiosity, did Mystic ever support the Hudson message bases in the past? I'd think that would have been a pain, trying to juggle three different formats.

No, it did not. The reason it did not is (if my memory serves me) was because Hudson had way too many technical limitations by the time Mystic came into play.

The earliest versions of Mystic (pre 0.01 circa 1995) had its own message base format that did on-the-fly LZH compression, which was a neat idea at the
time. Back then hard drive space actually mattered so LZH compressed bases could save you quite a bit!

I ditched that for JAM and Squish, and then eventually ditched Squish.

--- Mystic BBS v1.12 A35 (Windows/32)
* Origin: Sector 7 [Mystic BBS WHQ] (1:129/215)

On 06/25/17, g00r00 said the following...

It is a lot better. It was taking between 70-100 seconds to load unde A33. With A34, that time is down to 30-40 seconds. Great work!

And that should hopefully be reduced again to 1-2 seconds now in A35.

If A35 loads in 1-2 seconds, I will buy you a drink... Or 6... :)

No, it did not. The reason it did not is (if my memory serves me) was because Hudson had way too many technical limitations by the time Mystic came into play.

I was just wondering, as it seemed as though a lot of the programs tried to support everything at the time.

Hudson really did have limitations. I only used it for a couple message bases back in the 90s. Jam was what I used primarily.

I never even tried Squish. I don't remember why... I hate getting old... :)


---

Dan Richter
aka Black Panther
Sysop - Castle Rock BBS (RCS)
telnet://castlerockbbs.com
The sparrows are flying again...

--- Mystic BBS v1.12 A34 (Windows/64)
* Origin: Castle Rock BBS - castlerockbbs.com (1:317/3)

On 2017 Jun 24 23:11:00, you wrote to g00r00:

On a side note, stuff like this is the a benefit of ditching Squish
and only having JAM. It allows me to focus on coding things
optimally just for one specific format instead of coding to make
things work on many different systems at the cost of performance.

I can imagine that would help a lot. Just out of curiosity, did Mystic ever support the Hudson message bases in the past? I'd think that
would have been a pain, trying to juggle three different formats.

the original library viewed each message base type as an object... each message
base object did its thing as it needed to... instead of using each message base
object individually, you use a generic message base object... this generic message base object is simply derived from the message base object for the current message base format... the generic message base object simply hands everything to the current message base object and lets it have a go... if you change message bases and the new one is of a different format, you simply dispose of the generic message base object and initialize it again using the message base object for the new message base... now everything is done in the new message base's format... change bases again? dispose of the generic object and init it again with the new message base's object... you just always use the
generic message base object and the others are just black boxes... you feed data in for storage and you request data out for use...

no one ever needs to mess around in the deep guts of the library (other than to
fix or track down possible bugs or oversights)... all the calls in the generic message object are the same... trust me, i know because i use the same library and i don't have any problems writting to any of the formats supported... it is
great and is one of the biggest features of object programming!

i have, however, done my digging in the guts and yes, fixing some things was easily done... other things not so much... one example being the dates as stored in the various message base formats... one doesn't store seconds at all,
another stores only even seconds, the rest store all available seconds... using
this black box object stuff, you just send exactly the same date string and the
actual message base object will handle the seconds according to their format...

)\/(ark

Always Mount a Scratch Monkey
Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
... Food is an important part of a balanced diet.
---
* Origin: (1:3634/12.73)

g00r00 wrote to Dan Richter <=-

No, it did not. The reason it did not is (if my memory serves me) was because Hudson had way too many technical limitations by the time
Mystic came into play.

I can certainly understand that. I started out with RA 1.1 and the Hudson messagebase. I remember there was a size limit of 16 MB, which if exceeded would lead to severe corruption, causing all sorts of random crossposts. There was also a limit on the number of messages, IIRC. Hudson was a real cow to manage - miss a pack, or get your maintenance settings wrong and it was so easy to trash the messagebase on a busy board, because of these limitations. In the early 90s, this format was already past its useby date.

When JAM came along with RA 2.0, it was a breath of fresh air. I migrated most of my message areas to JAM and available disk space became the bigger concern (finally!).

The earliest versions of Mystic (pre 0.01 circa 1995) had its own
message base format that did on-the-fly LZH compression, which was a
neat idea at the time. Back then hard drive space actually mattered so LZH compressed bases could save you quite a bit!

Yes, that would have been handy in the early days!


... Real knowledge is to know the extent of ones ignorance.
--- MultiMail/Win32 v0.49
* Origin: Freeway BBS - freeway.apana.org.au (3:633/410)

On 06/25/17, g00r00 pondered and said...

I'll send it over for a testing if you'd like to give it a go.

Sounds good, I can certainly flood the BBS with lots of Usenet groups to push it.

--- Mystic BBS v1.12 A33 (Windows/32)
* Origin: Agency BBS | telnet://agency.bbs.geek.nz (3:770/100)