#saltr (irc2p) | I2P+ 2.8.5-7+ >> i2pplus.github.io | skank.i2p | git.skank.i2p | purokishi.i2p | vituperative.github.io/i2pchat | notbob.i2p | ramble.i2p | cake.i2p | shinobi.i2p | wall.i2p | /msg an op for voice

StormyCloud Started playing with it a little bit, just raw stats have not made anything look pretty yet cake.i2p/file/SQdCGmc5sn_56iR8Ukpv7mo4uq9zDYBHZDfcUboVH_Qgh4gMtjte/SCR-20250428-sgte.png

StormyCloud zzz any plans to be able to set the interface on the exporter plugin? Being able to send this data to our giant prometheus server would be more ideal.

zzz actually StormyCloud my question was about grafana on your website

zzz but what does 'set the interface' mean?

StormyCloud You can ping me the question if you want and I mean the listening interface. Right now it defaults to 127.0.0.1

zzz StormyCloud, it's running as a webapp on the console

zzz so it has the same iterface, port, and password if any

zzz you can change your console iterface on /configclients, but beware of the security risks

zzz my question was whether your outproxy stats are flopped, they show more outbound traffic than in

zzz or perhaps it's from the i2p-side perspective, but that seems wrong

zzz my other question was when stormycloud.i2p and your floodfill fleet were coming back

zzz how do you monitor your routers now? do you ssh in to each box and ssh tunnel the console port?

StormyCloud Alright I’ll take a look and let me double check the dashboard to make sure I didn’t transpose anything.

StormyCloud So grafana dashboards are good, these graphs are built just calculating the OS network traffic. As for stormycloud.i2p that node was acting funny so I shut it down, Ill make sure I get it back up today.

zzz I just keep ssh tunnels up to my hosted boxes and set the prometheus hostmane to localhost

zzz check prometheus first

StormyCloud We also have 40 floodfills, do we need more? Im happy to build more out.

zzz I only see three

StormyCloud But yes I ssh tunnel into each box, I have a remote management tool that makes it less painful

zzz in the migration did you rebuild your routers and not set the family?

zzz I've only seen three for about a month?

StormyCloud very odd, I just checked 4 random they are all showing floodfill on and part of the family

zzz hmm

zzz I wonder if they all got banned somehow?

StormyCloud I’ll shoot you the ips

zzz so they look happy?

StormyCloud I’m seeing traffic, tunnels, green status message

zzz yeah pm me the /24

zzz ok. I see a couple of them but that's it, I would expect to see more, I'll do some more investigation

zzz nothing obviously wrong, lets investigate further

zzz dr|z3d, how many stormies do you see from there?

zzz StormyCloud, back to the graphs, check to see if prometheus sees the routers

StormyCloud All Prometheus targets are up, but they stem just using base node exporter for stats. These were built before I2P got the exporter

StormyCloud Are just using*

zzz ok so you need to add another "target" for the plugin, with the console port and the metrics_path /prometheus/metrics

StormyCloud Yeah I started playing with the new stats last night to see what all we have. I got it working on a router but was unsure about changing the listening address last night

zzz I'm going to cut a 0.4 plugin right now

StormyCloud Roger that I’ll update this evening and continue playing with the stats

zzz with whatever your fancy management lets you get to the console, then you can get to the plugin

zzz it's just another url on the console

zzz postman, prometheus plugin updated to 0.4 to remove the state=average labels

zzz StormyCloud, did you rebuild your fleet when you migrated, or do they have the same identities as before?

zzz the good news is I don't see any IP bans for your /24

StormyCloud Yeah I’m able to see everything locally but typically I feed all the data into a giant Prometheus server. So having something like the tunnels where you could select a listening IP would be awesome.

StormyCloud They were rebuilt

zzz are they plus or canon?

StormyCloud Plus

StormyCloud dr|z3d: beats me if I select anything else

zzz StormyCloud, the suspicious thing is that you told me they were in the range a.b.c.x to a.b.c.y, and the only one I see is a.b.c.x

zzz perhaps you did some config copying that made all the rest not working right?

zzz I'd look closely to see if there's something different about the first one from all the rest

zzz as far as happiness

StormyCloud Ill dig into things this evening to see what is up

zzz I'm consistenty seeing 59 "NiggaNetworks" routers and 3 of your family, and most of the "NN" routers are on a single /24 also

StormyCloud Very interesting name…

zzz ok thanks. it seems to be pointing to your end, but I suggest you try to figure it out, not just mash the rebuild button

StormyCloud Hopefully they’re not on my /24 lol

zzz yeah we haven't discussed "NN" here before, but they've been around for a while, all i2pd, mostly on a /24 in Russia but some others scattered around the world

zzz maybe orignal knows something about them

StormyCloud Right, but yeah, I’ll do some digging and I’ll let you know what I find

zzz thanks

dr|z3d zzz: 2 right now

zzz something's not right

dr|z3d Maybe we've got a case of cloned router ids and stormy forgot to generate new keys? dunno

dr|z3d router.rebuildKeys=true in your config file or on /configadvanced and then restart router, StormyCloud?

StormyCloud That’s how they were rebuilt but I can try again this evening

zzz looking...

zzz yeah confirmed on two different routers I have two different versions of the router hash claiming to be at .36

zzz one is firewalled, one is not, and they have different static keys in their transports

zzz not quite right

zzz the routers at .36 and .37 have the same router hash. The one on .36 is ok, the one on .37 is firewalled

zzz I do suspect that all 40 of them have the same router hash, that's why we only see one at a time

zzz if that's how you rekeyed, it didn't work, so do it some other way

dr|z3d it should work, I suspect he forgot to rekey.

dr|z3d StormyCloud: try with 37, make a note of the hash on /info, add the config, reboot router, then verify the hash is different on /info after a restart.

dr|z3d let us know if it isn't, or indeed if it is.

zzz I can log a big fat error in the transports but will we connect to routers in our own /16?

zzz I think so but we won't build tunnels through them

zzz so lets add that log

StormyCloud Sounds good I’ll ping with an update tonight

StormyCloud Also rude dr|z3d I rebuilt the keys on deployment. You have taught me better

dr|z3d LOL. I should hope so, StormyCloud, but maybe you got distracted playing around with ComfyUI :)

StormyCloud Only thing I’ve been playing around with lately is Sharepoint X^X

dr|z3d I know RN has.. :)

cumlord interesting i used to see like 20 before now i only see 2 stormies

zzz it's a clone army

zzz working on adding checks and logs in 5 different places if you see an apparent clone

cumlord oh i see now was missing some scrollback

StormyCloud It’s just a very very multihomed router. Redundancy and all

orignal no I don't

orignal I'm wondering if these Russian IPs are from Novosibirsk ))

zzz the "NN" family?

orignal yes

orignal *** suspects plaz aka snowflakes here ***

orignal becuase he mentioned some bot for I2P recently

zzz I assume they are not attackers if they set a family, but maybe you can ask around. 60 routers is a lot

orignal But they put family NiggaNtwrok as insultation

orignal so Plazish ))

zzz about 2/3 of them are in 176.98.182.0/24

zzz the rest are all over

orignal but they are not Russia

zzz my geoip says that /24 is ru, but could be wrong

zzz StormyCloud, when you do rebuild them, please do all 40, not just 39, let that router hash die

zzz also, if you change the port on all of them, that will probably help things recover

dr|z3d most of the tm botnet should now be blocked in the latest + dev build.

zzz ok I have a collection of changes that will catch routers that appear to have your keys (encryption or signing or transport)

zzz the action it takes depends on whether they actually have your private keys, as proven by a signature or a noise handshake

zzz if they do, it will be a CRIT log

zzz saying you either cloned yourself or you've been hacked

zzz not a bad day for fishing, caught both a botnet and a clone army ))

zzz hmm actually I think all of the checks I just wrote are useless in this case

zzz because we'll never try to connect to ourselves or accept a store of our own RI

zzz will have to think harder about it

dr|z3d *clown army

dr|z3d *** winks at StormyCloud ***

orignal I would also ban all Baltic states

orignal because another kind of donkeyfuckers

StormyCloud zzz sounds good, it is a router identity issue. The router.rebuildKeys=true is at the top of the router.config in my deployment package so not sure what went wrong but it will be fixed today.