Digital Man wrote to Rmh <=-

Re: Messing around with Message Areas
By: Rmh to All on Fri Jul 13 2018 08:26 pm

I was importing a few thousand areas from Usenet in the believe all could just go well and when it was just about, I noticed, on loading the BBS, the messge pointers would be taking an eternity to load.

I've committed (to v3.17a in CVS) some significant optimizations to message pointer initializing (for new users or "guest") as well as the loading of message pointers during logon. I'd appreciate it if you
could try them out and see if it remedied your problem with importing thousands of newsgroups.

digital man

This Is Spinal Tap quote #35:
Jeanine Pettibone: You don't do heavy metal in Dubly, you know.
Norco, CA WX: 98.3oF, 34.0% humidity, 17 mph ENE wind, 0.00 inches rain/24hrs

---
= Synchronet = Vertrauen = Home of Synchronet = [vert/cvs/bbs].synchro.net

... Great beer bellies are made, not born.
--- MultiMail/Mac v0.51
* Synchronet * UNICYBER BBS -- bbs.unicyber.co.uk:2323::ssh bbs.unicyber.co.uk 2424
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Messing around with Messa
By: Rmh to Digital Man on Wed Jul 25 2018 06:14 am

I was importing a few thousand areas from Usenet in the believe all
could just go well and when it was just about, I noticed, on loading
the BBS, the messge pointers would be taking an eternity to load.

I've committed (to v3.17a in CVS) some significant optimizations to
message pointer initializing (for new users or "guest") as well as
the loading of message pointers during logon. I'd appreciate it if
you could try them out and see if it remedied your problem with
importing thousands of newsgroups.

I just performed a run on it.
So, I have upgraded to the latest :

src/sbbs3/ctrl/sbbsctrl.bpr:ProductVersion=3.17a

and loaded nearly 2000 Usenet Areas. (still got pending around 18000 for Alt + a few other optional thousands.. --no joke there, it's just massive--)

I have logged in a couple of times, and it takes around 2 minutes the Initialization for the message pointers.

I cannot see errors during the laod, or anything on the side, but if you want me doing anything else, please, let me know. I will be happy to hear suggestions.

I got the Guest account disabled, but feel free to log in and do a run around and if you need anything at all, let me know.

Many thanks Digital Man.

RainManHood<---

---
* Synchronet * UNICYBER BBS -- bbs.unicyber.co.uk:2323::ssh bbs.unicyber.co.uk 2424
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Messing around with Messa
By: Rmh to Digital Man on Wed Jul 25 2018 07:54 am

Re: Messing around with Messa
By: Rmh to Digital Man on Wed Jul 25 2018 06:14 am

I was importing a few thousand areas from Usenet in the believe all
could just go well and when it was just about, I noticed, on loading
the BBS, the messge pointers would be taking an eternity to load.

I've committed (to v3.17a in CVS) some significant optimizations to
message pointer initializing (for new users or "guest") as well as
the loading of message pointers during logon. I'd appreciate it if
you could try them out and see if it remedied your problem with
importing thousands of newsgroups.

I just performed a run on it.
So, I have upgraded to the latest :

src/sbbs3/ctrl/sbbsctrl.bpr:ProductVersion=3.17a

and loaded nearly 2000 Usenet Areas. (still got pending around 18000 for Alt + a few other optional thousands.. --no joke there, it's just massive--)

I have logged in a couple of times, and it takes around 2 minutes the Initialization for the message pointers.

That does seem excessive for only 2000 areas.

I cannot see errors during the laod, or anything on the side, but if you want me doing anything else, please, let me know. I will be happy to hear suggestions.

I got the Guest account disabled, but feel free to log in and do a run around and if you need anything at all, let me know.

Yes, please send me a copy of your data/user/0001.subs file.

digital man

Synchronet "Real Fact" #97:
Synchronet v3.12a was released on December 31st of 2004 (Rob's birthday). Norco, CA WX: 72.4oF, 76.0% humidity, 0 mph S wind, 0.00 inches rain/24hrs
--- SBBSecho 3.05-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Messing around with Messa
By: Digital Man to Rmh on Wed Jul 25 2018 03:03 am

Re: Messing around with Messa
By: Rmh to Digital Man on Wed Jul 25 2018 07:54 am

Re: Messing around with Messa
By: Rmh to Digital Man on Wed Jul 25 2018 06:14 am

I was importing a few thousand areas from Usenet in the believe all
could just go well and when it was just about, I noticed, on loading
the BBS, the messge pointers would be taking an eternity to load.

I've committed (to v3.17a in CVS) some significant optimizations to
message pointer initializing (for new users or "guest") as well as
the loading of message pointers during logon. I'd appreciate it if
you could try them out and see if it remedied your problem with
importing thousands of newsgroups.

I just performed a run on it.
So, I have upgraded to the latest :

src/sbbs3/ctrl/sbbsctrl.bpr:ProductVersion=3.17a

and loaded nearly 2000 Usenet Areas. (still got pending around 18000 for Alt + a few other optional thousands.. --no joke there, it's just massive--)

I have logged in a couple of times, and it takes around 2 minutes the Initialization for the message pointers.

That does seem excessive for only 2000 areas.

I cannot see errors during the laod, or anything on the side, but if you want me doing anything else, please, let me know. I will be happy to hear suggestions.

I got the Guest account disabled, but feel free to log in and do a run around and if you need anything at all, let me know.

Yes, please send me a copy of your data/user/0001.subs file.

Okay, I got the file. But my calculations, it contains 9705 message areas, not 2000. If you're running the latest v3.17a development code, you should see a debug-level log message whenever the msg scan config/pointers are loaded via the Terminal server, like this:
<username> read x message scan cfg/pointers in n ms

Can you share that log line with us (enable debug level log output if needed)?

digital man

This Is Spinal Tap quote #10:
Dozens of people spontaneously combust each year... just not widely reported. Norco, CA WX: 75.1oF, 73.0% humidity, 3 mph ESE wind, 0.00 inches rain/24hrs --- SBBSecho 3.05-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Messing around with Messa
By: Digital Man to Rmh on Wed Jul 25 2018 08:29 am

I have logged in a couple of times, and it takes around 2 minutes
the Initialization for the message pointers.

That does seem excessive for only 2000 areas.

Just a note on what I have seen here. Over the last week or so the scan has been getting faster and faster. At this point I don't even see it.

Of course I only have ~370 areas in my scan here.

Given time that may increase a bit if I can find interesting things to connect too.. :)

Ttyl :-),
Al


... And now for something you'll really like! -Rocky

---
* Synchronet * The Rusty MailBox - Penticton, BC Canada
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Messing around with Messa
By: Al to Digital Man on Wed Jul 25 2018 11:32 am

Re: Messing around with Messa
By: Digital Man to Rmh on Wed Jul 25 2018 08:29 am

I have logged in a couple of times, and it takes around 2 minutes
the Initialization for the message pointers.

That does seem excessive for only 2000 areas.

Just a note on what I have seen here. Over the last week or so the scan has been getting faster and faster. At this point I don't even see it.

Good to hear. I'm happy there were optimizations available to me.

Of course I only have ~370 areas in my scan here.

That's about 3 times the number I have currently. :-)

Given time that may increase a bit if I can find interesting things to connect too.. :)

Cool, it's all good test cases.

digital man

This Is Spinal Tap quote #14:
The Boston gig has been cancelled. [Don't] worry, it's not a big college town. Norco, CA WX: 93.2oF, 46.0% humidity, 9 mph E wind, 0.00 inches rain/24hrs
--- SBBSecho 3.05-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Messing around with Messa
By: Digital Man to Rmh on Wed Jul 25 2018 08:29 am

Okay, I got the file. But my calculations, it contains 9705 message areas, not 2000. If you're running the latest v3.17a development code, you should see a debug-level log message whenever the msg scan config/pointers are loaded via the Terminal server, like this:
<username> read x message scan cfg/pointers in n ms

Can you share that log line with us (enable debug level log output if needed)?

yes of course....

Jul 25 19:20:24 34819 sbbs: 7/25 19:20:24 term Node 1 received telnet cmd: DO Echo
Jul 25 19:20:24 34819 sbbs: 7/25 19:20:24 term Node 1 received telnet cmd: DO Suppress Go Ahead
Jul 25 19:20:24 34819 sbbs: 7/25 19:20:24 term Node 1 received telnet cmd: WILL Terminal Type
Jul 25 19:20:24 34819 sbbs: 7/25 19:20:24 term Node 1 requesting telnet terminal type
Jul 25 19:20:24 34819 sbbs: 7/25 19:20:24 term Node 1 received telnet cmd: WON'T Terminal Speed
Jul 25 19:20:24 34819 sbbs: 7/25 19:20:24 term Node 1 sending telnet cmd: DON'T Terminal Speed
Jul 25 19:20:24 34819 sbbs: 7/25 19:20:24 term Node 1 received telnet cmd: WON'T Send Location
Jul 25 19:20:24 34819 sbbs: 7/25 19:20:24 term Node 1 sending telnet cmd: DON'T Send Location
Jul 25 19:20:24 34819 sbbs: 7/25 19:20:24 term Node 1 received telnet cmd: WILL Negotiate About Window Size
Jul 25 19:20:24 34819 sbbs: 7/25 19:20:24 term Node 1 received telnet sub-negotiation command: Negotiate About Window Size
Jul 25 19:20:24 34819 sbbs: 7/25 19:20:24 term Node 1 received telnet window size: 136x37
Jul 25 19:20:24 34819 sbbs: 7/25 19:20:24 term Node 1 received telnet cmd: WILL New Environment Option
Jul 25 19:20:24 34819 sbbs: 7/25 19:20:24 term Node 1 received telnet sub-negotiation command: Terminal Type
Jul 25 19:20:24 34819 sbbs: 7/25 19:20:24 term Node 1 received telnet terminal type: pcansi
Jul 25 19:20:24 34819 sbbs: 7/25 19:20:24 term Node 1 received telnet cmd: WON'T Send Location
Jul 25 19:20:25 34819 sbbs: 7/25 19:20:25 term Node 1 received terminal auto-detection response: '\e[?1;2c\e[37;136R'
Jul 25 19:20:25 34819 sbbs: 7/25 19:20:25 term Node 1 received ANSI cursor position report: 136x37
Jul 25 19:20:33 34819 sbbs: 7/25 19:20:33 term Node 1 Rmh Rmh #1 System password verification failure
Jul 25 19:20:38 34819 sbbs: 7/25 19:20:38 term Node 1 Rmh Rmh #1 System password verification failure
Jul 25 19:21:26 34819 sbbs: 7/25 19:21:26 term Node 1 Rmh read 2326 message scan cfg/pointers in 40157 ms
Jul 25 19:21:26 34819 sbbs: 7/25 19:21:26 term Node 1 Rmh (0001) Rmh
Logon 0 - 37

Jul 25 19:21:29 34819 sbbs: 7/25 19:21:29 term Node 1 Rmh running external program: BullsEye! Bulletins

(A bit long, but I wanted to share the whole of it...)

RainManHood<---

---
* Synchronet * UNICYBER BBS -- bbs.unicyber.co.uk:2323::ssh bbs.unicyber.co.uk 2424
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Al wrote:

Re: Messing around with Messa
By: Digital Man to Rmh on Wed Jul 25 2018 08:29 am

I have logged in a couple of times, and it takes around 2 minutes
the Initialization for the message pointers.

That does seem excessive for only 2000 areas.

Just a note on what I have seen here. Over the last week or so the scan has been getting faster and faster. At this point I don't even see it.

Of course I only have ~370 areas in my scan here.

Given time that may increase a bit if I can find interesting things to

connect

too.. :)

I was having a conversation elsewhere with someone about this and mentioned it to
Digitalman. My ~1300 groups would take about 5-6 seconds to load the message pointers. Now it takes 151ms. You should definitely update to the latest from cvs.
--- SBBSecho 3.05-Linux
* Origin: End Of The Line BBS - endofthelinebbs.com (1:124/5016)

Re: Messing around with Messa
By: Rmh to Digital Man on Wed Jul 25 2018 07:31 pm

Can you share that log line with us (enable debug level log output if needed)?

Jul 25 19:21:26 34819 sbbs: 7/25 19:21:26 term Node 1 Rmh read 2326 message scan cfg/pointers in 40157 ms

Okay, that's 40 seconds (not 2 minutes), but still way too long to read those 2326 cfg/ptrs. But I think I know what's going on: your 0001.subs file still has the almost 10000 subs in it and that's make the search for the other/valid subs slower.

I just need to make the .subs file self-cleaning (removing invalid subs) when saved and that should fix you up.

digital man

Synchronet/BBS Terminology Definition #48:
SMTP = Simple Message Transfer Protocol
Norco, CA WX: 95.1oF, 40.0% humidity, 14 mph NE wind, 0.00 inches rain/24hrs --- SBBSecho 3.05-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Digital Man wrote to Rmh <=-

Re: Messing around with Messa
By: Rmh to Digital Man on Wed Jul 25 2018 07:31 pm

Can you share that log line with us (enable debug level log output if needed)?

Jul 25 19:21:26 34819 sbbs: 7/25 19:21:26 term Node 1 Rmh read 2326 message scan cfg/pointers in 40157 ms

Okay, that's 40 seconds (not 2 minutes), but still way too long to read those 2326 cfg/ptrs. But I think I know what's going on: your 0001.subs file still has the almost 10000 subs in it and that's make the search
for the other/valid subs slower.

I just need to make the .subs file self-cleaning (removing invalid
subs) when saved and that should fix you up.

Digital Man, how can I do that? What's the process?
I have been reading around the wiki, but I haven't found anything specific..

Many thanks.

... if ( original_ver == OK ) don't_upgrade();
--- MultiMail/Mac v0.51
* Synchronet * UNICYBER BBS -- bbs.unicyber.co.uk:2323::ssh bbs.unicyber.co.uk 2424
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Rmh wrote:

I just need to make the .subs file self-cleaning (removing invalid
subs) when saved and that should fix you up.

Digital Man, how can I do that? What's the process?
I have been reading around the wiki, but I haven't found anything specific..

0001.user is just a text file. If it was me I would make a backup and then you can delete all those you don't use or have been deleted, though it may take
a while to do that :) You might want to see if DM is able to make it retroactively delete them from that file.
--- SBBSecho 3.05-Linux
* Origin: End Of The Line BBS - endofthelinebbs.com (1:124/5016)

Digital wrote:

Re: Messing around with Messa
By: Rmh to Digital Man on Wed Jul 25 2018 07:31 pm

Can you share that log line with us (enable debug level log output if
needed)?

Jul 25 19:21:26 34819 sbbs: 7/25 19:21:26 term Node 1 Rmh read 2326 message
scan cfg/pointers in 40157 ms

Okay, that's 40 seconds (not 2 minutes), but still way too long to read

those

2326 cfg/ptrs. But I think I know what's going on: your 0001.subs file still has the almost 10000 subs in it and that's make the search for the

other/valid

subs slower.

I just need to make the .subs file self-cleaning (removing invalid subs)

when

saved and that should fix you up.

One thing I just posted today on IRC so you may not have seen it, and probably unrelated, is that when you delete an entire message group, the files are still left hanging around unlike when you delete a single group. You're prompted to remove all the related files. That means evertying will still be hanging around the filesystem if you don't manually delete them when removing a whole group.

---
* Synchronet * End Of The Line BBS - endofthelinebbs.com
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Messing around with Messa
By: Rmh to Digital Man on Thu Jul 26 2018 02:56 am

Digital Man wrote to Rmh <=-

Re: Messing around with Messa
By: Rmh to Digital Man on Wed Jul 25 2018 07:31 pm

Can you share that log line with us (enable debug level log output if needed)?

Jul 25 19:21:26 34819 sbbs: 7/25 19:21:26 term Node 1 Rmh read 2326 message scan cfg/pointers in 40157 ms

Okay, that's 40 seconds (not 2 minutes), but still way too long to read those 2326 cfg/ptrs. But I think I know what's going on: your 0001.subs file still has the almost 10000 subs in it and that's make the search for the other/valid subs slower.

I just need to make the .subs file self-cleaning (removing invalid subs) when saved and that should fix you up.

Digital Man, how can I do that? What's the process?
I have been reading around the wiki, but I haven't found anything specific..

Just update to the latest from CVS and when you logoff, your .subs file should get cleaned-up automatically.

digital man

Synchronet "Real Fact" #58:
The last version of Synchronet to run on MS-DOS and OS/2 was v2.30c (1999). Norco, CA WX: 73.7oF, 78.0% humidity, 0 mph SSW wind, 0.00 inches rain/24hrs --- SBBSecho 3.05-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Digital Man wrote to Rmh <=-

Re: Messing around with Messa
By: Rmh to Digital Man on Thu Jul 26 2018 02:56 am

Okay, that's 40 seconds (not 2 minutes), but still way too long to read those 2326 cfg/ptrs. But I think I know what's going on: your 0001.subs file still has the almost 10000 subs in it and that's make the search for the other/valid subs slower.

I just need to make the .subs file self-cleaning (removing invalid subs) when saved and that should fix you up.

Digital Man, how can I do that? What's the process?
I have been reading around the wiki, but I haven't found anything specific..

Just update to the latest from CVS and when you logoff, your .subs file should get cleaned-up automatically.

I just did (in case I am missing something...):

export CVSROOT=:pserver:anonymous@cvs.synchro.net:/cvsroot/sbbs
cvs update -d src 3rdp
cd /sbbs/src/sbbs3; make RELEASE=1 symlinks
cd /sbbs/src/sbbs3/scfg; make RELEASE=1 symlinks

I was logged on from the Terminal Server, still I tried twice log-in/log-off.

But I cannot appreciate any change. Pointers still taking time to load up.

Am I missing something??

Thanks Digital Man.


... To hell with criticism. Praise is good enough for me.
--- MultiMail/Mac v0.51
* Synchronet * UNICYBER BBS -- bbs.unicyber.co.uk:2323::ssh bbs.unicyber.co.uk 2424
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Messing around with Messa
By: Rmh to Digital Man on Thu Jul 26 2018 09:47 am

Okay, that's 40 seconds (not 2 minutes), but still way too long to
read those 2326 cfg/ptrs. But I think I know what's going on: your
0001.subs file still has the almost 10000 subs in it and that's
make the search for the other/valid subs slower.

I just need to make the .subs file self-cleaning (removing invalid
subs) when saved and that should fix you up.

Digital Man, how can I do that? What's the process?
I have been reading around the wiki, but I haven't found anything
specific..

Just update to the latest from CVS and when you logoff, your .subs
file
should get cleaned-up automatically.

I just did (in case I am missing something...):

export CVSROOT=:pserver:anonymous@cvs.synchro.net:/cvsroot/sbbs
cvs update -d src 3rdp

All good now!
I refreeshed code 2 minytes aog and the change are now there.
So all up to date and with the changes on the pointers fixed!!

Will get back with more informationas I will be uploading a few thousands more.

Thanks Digital Man.

RainManHood<---

---
* Synchronet * UNICYBER BBS -- bbs.unicyber.co.uk:2323::ssh bbs.unicyber.co.uk 2424
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Messing around with Messa
By: Rmh to Digital Man on Thu Jul 26 2018 02:00 pm

All good now!
I refreeshed code 2 minytes aog and the change are now there.
So all up to date and with the changes on the pointers fixed!!

Okay, cool. Can you paste the log line with the relevant stats now?

digital man

Synchronet/BBS Terminology Definition #32:
JS = JavaScript
Norco, CA WX: 91.0oF, 49.0% humidity, 10 mph NNE wind, 0.00 inches rain/24hrs --- SBBSecho 3.05-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Messing around with Messa
By: Digital Man to Rmh on Thu Jul 26 2018 02:54 pm

All good now!
I refreeshed code 2 minytes aog and the change are now there.
So all up to date and with the changes on the pointers fixed!!

Okay, cool. Can you paste the log line with the relevant stats now?

Jul 27 06:32:17 34819 sbbs: 7/27 06:32:17 term Node 1 received telnet terminal type: ANSI
Jul 27 06:32:17 34819 sbbs: 7/27 06:32:17 term Node 1 received telnet sub-negotiation command: Terminal Speed
Jul 27 06:32:17 34819 sbbs: 7/27 06:32:17 term Node 1 received telnet terminal speed: 38400,38400
Jul 27 06:32:18 34819 sbbs: 7/27 06:32:18 term Node 1 received terminal auto-detection response: '\e[?1;2c\e[53;157R'
Jul 27 06:32:18 34819 sbbs: 7/27 06:32:18 term Node 1 received ANSI cursor position report: 157x53
Jul 27 06:32:27 34819 sbbs: 7/27 06:32:27 term Node 1 Rmh read 345 message scan cfg/pointers in 7 ms
Jul 27 06:32:27 34819 sbbs: 7/27 06:32:27 term Node 1 Rmh (0001) Rmh
Logon 0 - 3


Now this seems really good.
Further testing on-going and opening a new Post which goes along with all this as I have found some strange thing related to the import of a backbone style file..

Thanks Digital Man

RainManHood<---

---
* Synchronet * UNICYBER BBS -- bbs.unicyber.co.uk:2323::ssh bbs.unicyber.co.uk 2424
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Messing around with Messa
By: Rmh to Digital Man on Fri Jul 27 2018 06:42 am

Okay, cool. Can you paste the log line with the relevant stats now?

Jul 27 06:32:27 34819 sbbs: 7/27 06:32:27 term Node 1 Rmh read 345 message scan cfg/pointers in 7 ms

Um... what happened to the thousands of message areas? :-(

digital man

This Is Spinal Tap quote #45:
I don't really think the end can be assessed as of itself as being the end Norco, CA WX: 68.3oF, 89.0% humidity, 0 mph S wind, 0.00 inches rain/24hrs
--- SBBSecho 3.05-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Digital Man wrote to Rmh <=-

Re: Messing around with Messa
By: Rmh to Digital Man on Fri Jul 27 2018 06:42 am

Okay, cool. Can you paste the log line with the relevant stats now?

Jul 27 06:32:27 34819 sbbs: 7/27 06:32:27 term Node 1 Rmh read 345 message scan cfg/pointers in 7 ms

Um... what happened to the thousands of message areas? :-(

No idea. really. I didn't delete/remove anything at the time of the test.
Do you want me trying anything else?



... HA HAH HA HAH HAH <SMACK> ... oof ...
--- MultiMail/Mac v0.51
* Synchronet * UNICYBER BBS -- bbs.unicyber.co.uk:2323::ssh bbs.unicyber.co.uk 2424
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Messing around with Messa
By: Rmh to Digital Man on Fri Jul 27 2018 10:15 am

Digital Man wrote to Rmh <=-

Re: Messing around with Messa
By: Rmh to Digital Man on Fri Jul 27 2018 06:42 am

Okay, cool. Can you paste the log line with the relevant stats now?

Jul 27 06:32:27 34819 sbbs: 7/27 06:32:27 term Node 1 Rmh read 345 message scan cfg/pointers in 7 ms

Um... what happened to the thousands of message areas? :-(

No idea. really. I didn't delete/remove anything at the time of the test.
Do you want me trying anything else?

According the logs snippets you sent:

Jul 25 19:21:26 34819 sbbs: 7/25 19:21:26 term Node 1 Rmh read 2326 message scan cfg/pointers in 40157 ms
Jul 27 06:32:27 34819 sbbs: 7/27 06:32:27 term Node 1 Rmh read 345 message scan cfg/pointers in 7 ms

You removed about 2000 message areas (sub-boards) from your configuration (SCFG->Message Areas) between 7/25 and 7/27.

digital man

This Is Spinal Tap quote #39:
Airport Security Officer: Do you have any artificial plates or limbs?
Norco, CA WX: 84.6oF, 58.0% humidity, 6 mph NE wind, 0.00 inches rain/24hrs
--- SBBSecho 3.05-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Messing around with Messa
By: Digital Man to Rmh on Fri Jul 27 2018 10:45 am

According the logs snippets you sent:

Jul 25 19:21:26 34819 sbbs: 7/25 19:21:26 term Node 1 Rmh read 2326 message scan cfg/pointers in 40157 ms
Jul 27 06:32:27 34819 sbbs: 7/27 06:32:27 term Node 1 Rmh read 345 message scan cfg/pointers in 7 ms

You removed about 2000 message areas (sub-boards) from your configuration (SCFG->Message Areas) between 7/25 and 7/27.

I can't remember removing anything in between.
Today for example I did and I tried to re-upload everything again.

Nevertheless I can run more testing on it.

RainManHood<---

---
* Synchronet * UNICYBER BBS -- bbs.unicyber.co.uk:2323::ssh bbs.unicyber.co.uk 2424
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Messing around with Messa
By: Rmh to Digital Man on Fri Jul 27 2018 06:53 pm

Re: Messing around with Messa
By: Digital Man to Rmh on Fri Jul 27 2018 10:45 am

According the logs snippets you sent:

Jul 25 19:21:26 34819 sbbs: 7/25 19:21:26 term Node 1 Rmh read 2326 message scan cfg/pointers in 40157 ms
Jul 27 06:32:27 34819 sbbs: 7/27 06:32:27 term Node 1 Rmh read 345 message scan cfg/pointers in 7 ms

You removed about 2000 message areas (sub-boards) from your configuration (SCFG->Message Areas) between 7/25 and 7/27.

I can't remember removing anything in between.
Today for example I did and I tried to re-upload everything again.

Well you can look at your ctrl/msgs.*.cnf backup files and see if/when the sizes changed dramatically.

digital man

Synchronet "Real Fact" #87:
Synchronet Message Base and its library, SMBLIB, was used by AXiS BBS Software. Norco, CA WX: 95.5oF, 37.0% humidity, 6 mph E wind, 0.00 inches rain/24hrs
--- SBBSecho 3.05-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Messing around with Messa
By: Digital Man to Rmh on Fri Jul 27 2018 10:45 am

Re: Messing around with Messa
By: Rmh to Digital Man on Fri Jul 27 2018 06:42 am

Okay, cool. Can you paste the log line with the relevant stats
now?

Jul 27 06:32:27 34819 sbbs: 7/27 06:32:27 term Node 1 Rmh read 345
message scan cfg/pointers in 7 ms

Um... what happened to the thousands of message areas? :-(

No idea. really. I didn't delete/remove anything at the time of the
test. Do you want me trying anything else?

According the logs snippets you sent:

Jul 25 19:21:26 34819 sbbs: 7/25 19:21:26 term Node 1 Rmh read 2326 message scan cfg/pointers in 40157 ms
Jul 27 06:32:27 34819 sbbs: 7/27 06:32:27 term Node 1 Rmh read 345 message scan cfg/pointers in 7 ms

You removed about 2000 message areas (sub-boards) from your configuration (SCFG->Message Areas) between 7/25 and 7/27.

Digital Man,
thanks for fixing the mass-load. I can get everything in now.
Further tests as below:

After a successful Mass-load for Usenet Echo(es) from 0-B which is a bit more than 10.000 (many thanks as usual Digital Man):


Jul 29 16:58:38 34819 sbbs: 7/29 16:58:38 term Node 1 received telnet sub-negotiation command: Terminal Speed
Jul 29 16:58:38 34819 sbbs: 7/29 16:58:38 term Node 1 received telnet terminal speed: 38400,38400
Jul 29 16:58:39 34819 sbbs: 7/29 16:58:39 term Node 1 received terminal auto-detection response: '\e[?1;2c\e[53;157R'
Jul 29 16:58:39 34819 sbbs: 7/29 16:58:39 term Node 1 received ANSI cursor position report: 157x53
Jul 29 16:58:48 34819 sbbs: 7/29 16:58:48 term Node 1 Rmh Node 1 Time reduced (to 12:01:12) due to upcoming event (msgmaint) on Mon Jul 30 2018 05:00 am
Jul 29 16:59:24 34819 sbbs: 7/29 16:59:24 term Node 1 Rmh read 10592 message scan cfg/pointers in 36186 ms
Jul 29 16:59:24 34819 sbbs: 7/29 16:59:24 term Node 1 Rmh (0001) Rmh
Logon 0 - 5

If I try to send a netmail I get :


!JavaScript /sbbs/exec/SlyEdit_Misc.js line 3240: out of memory

Aborted.


Also :

l 29 17:02:28 34819 sbbs: 7/29 17:02:28 evnt Timed event: LISTSERV returned 0 Jul 29 17:02:28 34819 sbbs: 7/29 17:02:28 evnt Running timed event: AVAT-IN
Jul 29 17:02:28 34819 sbbs: 7/29 17:02:28 evnt AVAT-IN Executing external: ?avatars import
Jul 29 17:02:28 34819 sbbs: 7/29 17:02:28 evnt AVAT-IN !JavaScript /sbbs/exec/load/syncdata.js line 10: out of memory
Jul 29 17:02:28 34819 sbbs: 7/29 17:02:28 evnt AVAT-IN !JavaScript /sbbs/exec/load/syncdata.js line 10: out of memory
Jul 29 17:02:28 34819 sbbs: 7/29 17:02:28 evnt Timed event: AVAT-IN returned 0 Jul 29 17:02:28 34819 sbbs: 7/29 17:02:28 evnt Running timed event: AVAT-OUT Jul 29 17:02:28 34819 sbbs: 7/29 17:02:28 evnt AVAT-OUT Executing external: ?avatars export
Jul 29 17:02:28 34819 sbbs: 7/29 17:02:28 evnt AVAT-OUT !JavaScript /sbbs/exec/load/cga_defs.js line 59: out of memory
Jul 29 17:02:28 34819 sbbs: 7/29 17:02:28 evnt AVAT-OUT !JavaScript /sbbs/exec/load/cga_defs.js line 59: out of memory
Jul 29 17:02:29 34819 sbbs: 7/29 17:02:29 evnt Timed event: AVAT-OUT returned 0 Jul 29 17:02:29 34819 sbbs: 7/29 17:02:29 evnt Running timed event: BINK-USE Jul 29 17:02:29 34819 sbbs: 7/29 17:02:29 evnt BINK-USE Executing external: ?binkit -l 1:229/426
Jul 29 17:02:29 34819 sbbs: 7/29 17:02:29 evnt BINK-USE !JavaScript /sbbs/exec/binkit.js line 20: out of memory
Jul 29 17:02:29 34819 sbbs: 7/29 17:02:29 evnt BINK-USE !JavaScript /sbbs/exec/binkit.js line 20: out of memory
Jul 29 17:02:29 34819 sbbs: 7/29 17:02:29 evnt Timed event: BINK-USE returned 0 Jul 29 17:02:46 34819 sbbs: 7/29 17:02:46 term Node 1 Rmh sending telnet cmd: GA
Jul 29 17:03:22 34819 sbbs: 7/29 17:03:22 term Node 1 Rmh disconnecting client Jul 29 17:03:23 34819 sbbs: 7/29 17:03:23 term Node 1 input thread terminated (received 162 bytes in 57 blocks)

And system memory:

$ free -m
total used free shared buff/cache available Mem: 4096 1582 119 27 2394 2217 Swap: 2048 216 1831



So, I nearly killed my VPS host....
I had to remove all the areas as it was really over the top...

As usual, thanks Digital Man.

RainManHood<---

---
* Synchronet * UNICYBER BBS -- bbs.unicyber.co.uk:2323::ssh bbs.unicyber.co.uk 2424
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Messing around with Messa
By: Rmh to Digital Man on Sun Jul 29 2018 05:25 pm

Re: Messing around with Messa
By: Digital Man to Rmh on Fri Jul 27 2018 10:45 am

Re: Messing around with Messa
By: Rmh to Digital Man on Fri Jul 27 2018 06:42 am

Okay, cool. Can you paste the log line with the relevant stats DM>> now?

Jul 27 06:32:27 34819 sbbs: 7/27 06:32:27 term Node 1 Rmh read 345
message scan cfg/pointers in 7 ms

Um... what happened to the thousands of message areas? :-(

No idea. really. I didn't delete/remove anything at the time of the
test. Do you want me trying anything else?

According the logs snippets you sent:

Jul 25 19:21:26 34819 sbbs: 7/25 19:21:26 term Node 1 Rmh read 2326 message scan cfg/pointers in 40157 ms
Jul 27 06:32:27 34819 sbbs: 7/27 06:32:27 term Node 1 Rmh read 345 message scan cfg/pointers in 7 ms

You removed about 2000 message areas (sub-boards) from your configuration (SCFG->Message Areas) between 7/25 and 7/27.

Digital Man,
thanks for fixing the mass-load. I can get everything in now.
Further tests as below:

After a successful Mass-load for Usenet Echo(es) from 0-B which is a bit more than 10.000 (many thanks as usual Digital Man):

Jul 29 16:59:24 34819 sbbs: 7/29 16:59:24 term Node 1 Rmh read 10592 message scan cfg/pointers in 36186 ms

36 seconds is a long time. But 10K+ message areas is a lot to keep track of, so I'm not too surprised.

Jul 29 16:59:24 34819 sbbs: 7/29 16:59:24 term Node 1 Rmh (0001) Rmh
Logon 0 - 5

If I try to send a netmail I get :

!JavaScript /sbbs/exec/SlyEdit_Misc.js line 3240: out of memory

You'd have to increase the size of JavaScript heap (in your ctrl/sbbs.ini file).

Jul 29 17:02:28 34819 sbbs: 7/29 17:02:28 evnt AVAT-IN !JavaScript /sbbs/exec/load/syncdata.js line 10: out of memory

Yup, same problem.

So, I nearly killed my VPS host....
I had to remove all the areas as it was really over the top...

I didn't really think it was going to work. :-)

digital man

Synchronet "Real Fact" #35:
The irc.synchro.net network has more servers than users.
Norco, CA WX: 74.9oF, 70.0% humidity, 0 mph S wind, 0.00 inches rain/24hrs
--- SBBSecho 3.05-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Digital Man wrote to Rmh <=-

According the logs snippets you sent:

Jul 25 19:21:26 34819 sbbs: 7/25 19:21:26 term Node 1 Rmh read 2326 message scan cfg/pointers in 40157 ms
Jul 27 06:32:27 34819 sbbs: 7/27 06:32:27 term Node 1 Rmh read 345 message scan cfg/pointers in 7 ms

You removed about 2000 message areas (sub-boards) from your configuration (SCFG->Message Areas) between 7/25 and 7/27.

Digital Man,
thanks for fixing the mass-load. I can get everything in now.
Further tests as below:

After a successful Mass-load for Usenet Echo(es) from 0-B which is a bit more than 10.000 (many thanks as usual Digital Man):

Jul 29 16:59:24 34819 sbbs: 7/29 16:59:24 term Node 1 Rmh read 10592 message scan cfg/pointers in 36186 ms

36 seconds is a long time. But 10K+ message areas is a lot to keep
track of, so I'm not too surprised.

Very much , yes.

Jul 29 16:59:24 34819 sbbs: 7/29 16:59:24 term Node 1 Rmh (0001) Rmh
Logon 0 - 5

If I try to send a netmail I get :

!JavaScript /sbbs/exec/SlyEdit_Misc.js line 3240: out of memory

You'd have to increase the size of JavaScript heap (in your
ctrl/sbbs.ini file).

Jul 29 17:02:28 34819 sbbs: 7/29 17:02:28 evnt AVAT-IN !JavaScript /sbbs/exec/load/syncdata.js line 10: out of memory

Thanks, I forgot that setting.

Yup, same problem.

So, I nearly killed my VPS host....
I had to remove all the areas as it was really over the top...

I didn't really think it was going to work. :-)

Anyhow it was worth to have a look at it. Probably you don't have all much testing these days, so We also now how far we can go with the system.
And, if I didn't try, I wouldn't know.

Your help is very much appreciated Digital Man.


... I got everything but the part after "Now listen closely".
--- MultiMail/Mac v0.51
* Synchronet * UNICYBER BBS -- bbs.unicyber.co.uk:2323::ssh bbs.unicyber.co.uk 2424
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

Re: Messing around with Messa
By: Rmh to Digital Man on Mon Jul 30 2018 06:29 pm

Anyhow it was worth to have a look at it. Probably you don't have all much testing these days, so We also now how far we can go with the system.
And, if I didn't try, I wouldn't know.

Your help is very much appreciated Digital Man.

Your testing and bug reports are are appreciated!

digital man

This Is Spinal Tap quote #45:
I don't really think the end can be assessed as of itself as being the end Norco, CA WX: 93.2oF, 35.0% humidity, 10 mph ENE wind, 0.00 inches rain/24hrs --- SBBSecho 3.05-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

On 2018 Jul 29 17:25:26, you wrote to Digital Man:

And system memory:

$ free -m
total used free shared buff/cache available
Mem: 4096 1582 119 27 2394
Swap: 2048 216 1831

damn, they only give you half your memory for swap? not too bad, though... you're only using 216 of it... waaaait, this came after the out of memory logs,
right? so things may have cleared out a bit...

)\/(ark

Always Mount a Scratch Monkey
Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
... You win a collection of ugly ASCII art for use in dumb sig lines.
---
* Origin: (1:3634/12.73)