IRCaBot 2.1.0
GPLv3 © acetone, 2021-2022
#i2p-dev
/2023/02/07
@eyedeekay
&kytv
&zzz
+R4SAS
+RN
+RN_
+dr|z3d
+orignal
+postman
+wodencafe
Arch
DeltaOreo
FreeRider
FreefallHeavens
Irc2PGuest19353
Irc2PGuest22478
Irc2PGuest48042
Irc2PGuest64530
Irc2PGuest77854
Nausicaa
Onn4l7h
Onn4|7h
Over1
Sisyphus
Sleepy
Soni
T3s|4_
Teeed
aargh3
acetone_
anon4
b3t4f4c3
bak83_
boonst
cumlord
dr4wd3
eyedeekay_bnc
hagen_
khb
not_bob_afk
plap
poriori
profetikla
r3med1tz
rapidash
shiver_1
solidx66
tr
u5657
uop23ip
w8rabbit
weko_
x74a6
zzz worked for me, as reported
dr|z3d yeah, minor in the scheme of things. restart of router will probably fix.
UDP Do you guys know if the wrapper.java.maxmemory flag in wrapper.config scales automatically with the bandwidth set upon first setup? I've had to modify it manually on a couple machines since I have a rather fast internet connection, and it would just keep crashing after 4-5 minutes.
obscuratus UDP: Yep, you have to set that manually in the wrapper.config
UDP Seems like it runs out of heap memory with the default value, so just wondering.
UDP ah gotcha, that's good to know since that doesn't seem to be covered on the website's basic install steps, and I was trying to figure it out :p
UDP thanks
UDP got 3-4 nodes running so far, one's done a few terabytes this week
UDP gonna go get some food, but gonna lurk on here a bit :)
Opicaak It appears like the network is back to normal, for now.
dr|z3d like I said to zzz earlier, Opicaak, just because the view from where you are looks ok, it doesn't mean that's the same everywhere. attack isn't hitting all routers at once.
eche|off hm, memory footprint did went up, some routers did died of 256 MB java heap memory as of OOM, to bad
zzz you'll be at the meeting today right eche|off ?
zzz first report of OOMs, thanks
eche|off with 2.0.1 it was.
zzz 2.1.0?
eche|off I try to be, need to check if I can get into with laptop. still system broken :_/
eche|off 2.1.0, latest ubunti
eche|off (sorry, currently no dev build until system is rebuild...)
zzz what else? fast router? slow? floodfill? any services on it?
eche|off 1 service on it, mde-fast router, defaultiung to 500kb/sec, else default, reachable via IPv4 and IPv6
eche|off 80% share
zzz ok. did it restart ok by itself and come back up?
eche|off by default ~2k active peers
eche|off it did restart itself ~10 times and went of later on
eche|off restart after 3-5 min of being active
eche|off mostly kademlia datastorejobs I see in the out of heap memory errors
zzz good to know, expected it but did not have any reports yet
zzz will have a package of fixes later in the week
zzz workaround 1, increase max mem in wrapper.config of course
zzz workaround 2, when stopped, just delete all but a thousand or so of your RIs
eche|off just 9k
eche|off in netDB
eche|off ah, 19k in peerProfile
eche|off so, default alue is bad in these situations
eche|off interstingly the bigger oney do have low numbers now, but guess they are rated low on most peers
eche|off which shows the net does work
zzz not_bob, did you resolve the snark recheck-every-startup issue from my forum and if so how? symlinks? plus bug?
dr|z3d pfft. plus bug my aspidistra.
dr|z3d that would be in the same category as the double leaseset I2P+ jeopardy bug.
dr|z3d aka canon bug.
not_bob zzz: Sometimes it does it now, and other times it does not. I have no idea why.
not_bob zzz: If I do a bad shutdown it does a re-checke very time.
not_bob It has nothing to do with symlinks.
dr|z3d running it with i2pd on the backend, no, not_bob?
dr|z3d could be session data's not being given enough time to write to disk before shutdown..
not_bob i2pd backend.
not_bob My fleet is mostly i2pd at this point due to low memory.
zzz ok, thx for update
zzz just asking
not_bob No worries. The re-checking at start isn't that big of a deal. It just takes a while.
not_bob I have just shy of 2 TB or torrents loaded on one router.
dr|z3d my money's on session data being corrupted on shutdown.
zzz probably a shutdown thing, yeah. not corruption, just not getting it all done
zzz before the router gives up waiting
dr|z3d right, same thing really, if the data's not all written in time, then on startup it's, well, corrupted :)
dr|z3d rotating the save points could mitigate that. so you maintain 2, current and last.
zzz not_bob, try stopping i2psnark at the same time you click graceful shutdown, should fix it
dr|z3d if current's not good, rollback to last.
not_bob Next time I restart that server I'll give it more time before I tell it to restart.
not_bob zzz: I'll try that and report back later.
zzz it's not corrupt drz, just didn't get the new one written, so the files are newer than the timestamp in the old one
dr|z3d why would files be newer if they're already downloaded and are seeding, assuming we're talking about the content here, and not the metadata?
zzz right, only if incomplete
dr|z3d so, not_bob's probably is a TB of completed torrents all wanting to do the integrity check dance on startup.
dr|z3d *problem
dr|z3d that's why I'm suggesting the config/metadata is being corrupted on shutdown.
not_bob Yeah, almost all of my torrents are done and just there to upload to others.
zzz it's also very hard to corrupt a config file because of the way we do it:
zzz save to x.tmp then rename to x
eyedeekay Hi everybody, sorry I'm late, got disconnected right before the meeting
eyedeekay 2. 2.1.0 Status Report
eyedeekay 3. 2.2.0 Development Status
eyedeekay 4. Congestion Throttling
eyedeekay 5. Hypothetical Traffic Management ( Flood of Tor Users)
eyedeekay 6. Stickers for translators
eyedeekay zzz where do you want to do your report about the DOS? 2 or own topic?
zzz let's call it 3b)
zzz or 2b)
zzz your choice
eyedeekay OK 2b then
eyedeekay 1. Hi who all is here today besides me and zzz?
echelonMAC on replaement system.
eyedeekay Great turnout, thanks everybody
zzz irc is laggier than usual so please allow a little extra time for responses
eyedeekay 2. 2.1.0 Status Report
eyedeekay Thanks zzz I will keep that in mind
eyedeekay Where to even start... my focus has been on Android UX issues in particular fixing up things I was doing wrong when initializing the app
eyedeekay Android will get a point release on that account
eyedeekay The topic I'm sure is on everybody's mind right now is the network being under attack which has shifted everybody's attention
eyedeekay And that is topic 2b
eyedeekay Shoot I mixed up the topics, scratch that, that was supposed to go to 3. sorry, long week, lot going on
eyedeekay zzz would you do me a favor to take 2? I think I am ill-equipped to present on everything which is going on
zzz sure
zzz but before I do
zzz do you and echelonMAC have a 2.1.0 mac easy-install bundle progress report?
eyedeekay Oh yes I can do that
echelonMAC in short: eyedeekaym did redo the signing script, I can buiodl and sign, but still wrong signing for apple
eyedeekay So, we've been closing in on issues, and right now echelonMAC and have different hypothesis on what's going on which we're both pursuing
eyedeekay My hypothesis is that this all stems from a stale workaround for a bug in Java 14
eyedeekay Where jpackage is supposed to sign the dylib files that are packed into a disk image(dmg) but which are in fact left unsigned
eyedeekay zab worked around this by creating the .app directory and signing the stuff inside it then using jpackage only for the final packaging step
eyedeekay so when we upgraded Java's we started signing everything twice, and we no longer needed to do that signing step manually
eyedeekay echelonMAC correct me if I'm wrong but you think you have the wrong sort of keys?
echelonMAC maybe, maybe not, unsure about that
echelonMAC at least the logs showing this error
zzz my question was higher level, what's the ETA, is 2.1.0 still a target or has that been abandoned and you're hoping for 2.2.0, or what?
echelonMAC 2.1.0 is stioll the target, but currently no ETA
echelonMAC I can buidl nearly instant, but digging deeper is currently out of time...
eyedeekay I still want to cut a release of 2.1.0 as soon as we know we can pass notarization, IMO once we can pass it once we should be able to do it predictably
zzz eyedeekay more or less committed to early april 2.2.0 in his blog post, you're not endorsing that ETA echelonMAC ?
echelonMAC I am completely helpless and cant predict a ETA, as I do not full yunderstand signing and building yet.
echelonMAC more or less, I do follow scripts blindly, if idk can fix the signage, the notarization is a 5 min job.
echelonMAC IF the sign does work as appple expect it
zzz ok then I recommend an edit of the blog post eyedeekay, let's not promise things we can't deliver on
eyedeekay Will do
zzz ok thanks you ready for my part of 2) ?
eyedeekay Yes please
zzz great
zzz last meeting was one week after the release, now we're 4 weeks out
zzz my hope was that expl. build success would climb steadily
zzz from the low-20% to low-30%, and then "pop" out of congestion, back to low 40s
zzz only the first part happened
zzz and then we swung back and forth between low 20s and low 30s
zzz so, we have some theories, see 2b)
zzz but I'm happy with the performance of 2.1.0 otherwise
zzz not too many bug reports
zzz I'll give an overview of what we are fixing in 2b) and 3)
zzz about 50% of the network has updated to 2.1.0 or the i2pd equivalent
zzz everybody please update if you haven't
zzz that's all I have for 2a), but I'll pause for a minute for questions / comments before starting 2b)
zzz ok, 2b) current network conditions
zzz over the weekend started an unambiguous attack
zzz lots of floodfill routers
zzz for the most part, the network overall, and java routers, are handling it ok
zzz I do have one report of routers crashing with OOM (out of memory)
zzz I understand that i2pd routers are really strugging with very low tunnel build success rates
not_bob My fleet is up to date.
zzz the attack is starting / stopping / changing several times a day
zzz so we're only about 60 hours in to understanding it and discussing countermeasures
zzz remember the tunnel build problem (now much more likely to be classified as a different attack) started Dec. 19 and took us several weeks to understand and address
zzz so it's early days
not_bob I have one I2P+ router and it's done well to weather this. But, my i2pd routers not so much. I've seen as low as 3% tunnel build success. I'm currently sitting around 10% on those routers.
zzz but this is a straightforward sybil / DHT attack, new to us but familiar to anybody doing peer-to-peer / DHT applications
zzz far too early to say if it will require a release ahead of schedule (for java i2p) or if so when
zzz EOT, I'll wait a couple minutes for discussion / comments / questions
eyedeekay Should people who wind up OOM increase the RAM available to their router?
zzz yeah, that's a straightforward mitigation
zzz stop your router, edit wrapper.config, restart
zzz I expect I'll have mitigations in dev builds in a few days
eyedeekay As I mistakenly stated before, my focus has been on Android UX issues in particular fixing up things I was doing wrong when initializing the app, pretty basic stuff all things considered but all of which was due for improvement
eyedeekay or outright fixing
eyedeekay Examples being fixes for subscriptions and custom ports when building tunnels which got in several user's way after a large increase in users with 2.10
dr|z3d as an aside, and to shine some light on the size of the attack, we've got a user reporting ~30K banned routers. so it's non-trivial :)
dr|z3d (that'll be shiver, who's here)
mark22k I got 56005 banned peers.
moristo Is this the work of a nation state--the banned routers or any other noticable patten?
eyedeekay Holy moley. I have 11027 and I thought that was a lot
moristo Spectrum internet was off yesetrday in FL and Italy the day before.
moristo *yesterday.
zzz let's get back to 3) please and table further attack discussion until after the meeting
echelonMAC Banned Peers (57053)
moristo oh, is there a meeting in progress? My bad.
zzz eyedeekay, you still with us?
eyedeekay yeah I'm here
zzz you have more on 3) or is it my turn?
eyedeekay started a long one:
eyedeekay i2p.firefox(Easy-Install for Windows is also getting worked on, more of the work is being done by removal there, updates are getting simplified now that the old admin-style installs are all migrated to user-style installs
eyedeekay portable USB install support is on the horizon for 2.2.0
eyedeekay With updates
eyedeekay Other issues and user-complaints I'm addressing or deciding how to address are on that issue tracker
eyedeekay EOT for me
zzz ok you saw the NPE issue in i2p.i2p right?
eyedeekay Yes I did, hot on the trail
zzz ok holler if you need help ofc
zzz 3) for me:
zzz I finished the peer selection refactor I've been working on since september, finally
zzz I think I fixed the erroneous symmetric nat indications that were in 2.1.0, but need somebody with that problem to test and report
zzz got a cool new i2psnark search box
zzz almost done with "congestion caps" (proposal 162)
zzz and some more tweaks to refine our handling of tunnel build congestion
zzz late last week, I thought I was pretty much done with 2.2.0 and could relax for two months until the release
zzz so, unfortunately, now we have a lot more to do
zzz that's the way it goes sometimes
not_bob Thank you for that, a major quality of life improvement.
zzz EOT, I'll wait a minute for discussion, then back to you eyedeekay
zzz haha not_bob you're welcome
eyedeekay Last call for 3?
eyedeekay 4. Congestion Throttling - I think this was a tunnel_king topic, is tunnel_king here?
zzz back to you eyedeekay
eyedeekay 4. Congestion Throttling, 5. Hypothetical Traffic Management ( Flood of Tor Users) - both introduced by tunnel_king, last call for tunnel_king
eyedeekay OK last one, 6. Stickers for translators
eyedeekay Specifically rules for people receiving stickers outside of the Americas
zzz this was my topic, only because unresolved since last meeting
zzz echelonMAC you have an answer?
echelonMAC not en detail, but who wnats should receive a bunch of stickers if they sent their address
echelonMAC aka sned a announcement in transifex and send out after receive of address
echelonMAC but currently no new stickers arrivced here
eyedeekay Tracking says the 10th
zzz I'm awaiting echelonMAC to post the instructions on my forum, then I will copy/paste to transifex
zzz that's where we've been for a month
zzz thanks
zzz EOT on 6) for me, back to you eyedeekay
eyedeekay Anything else for 6 or for the meeting?
zzz one closing word for the meeting: zzz.i2p the best place for news and weather, I'll post there when I know more than I do now
zzz thanks
eyedeekay Thanks very much for that zzz, and thanks everybody for coming to the meeting
zzz thanks eyedeekay
zzz got thru it without disconnects
zzz anybody with a registered nick that wants voice, speak up before I +m
eyedeekay See you around IRC and same time next month
justmessin unsure if I can speak in this channel, can anyone hear me? I was going to ask for the logs of this channel/details of the attack. I just recently started using I2P a few weeks ago but have a background in cybersecurity
obscuratus justmessin: Check major.i2p
not_bob justmessin: Logs will be posted on geti2p.net/en/meetings
eyedeekay justmessin you could look at the #ls2 channel logs on major.acetone.i2p
zzz last call for voice justmessin mark22k moristo or anybody else
justmessin hey zzz
zzz does hey mean you want voice?
justmessin could I get voice please
justmessin obscuratus not_bob eyedeekay thanks all I will check. would any devs find router stats valuable?
justmessin i just started recently with i2p this past month but i have everything open, max tunnels quite high etc
zzz welcome to i2p
zzz no we don't need stats from anybody
justmessin thank you, gonna get back to something but then i'll take a look through these logs to see if theres anything i can suggest
justmessin is this channel okay for questions about routers? i only just recently started hosting one. started with default java i2p default 1G, now using 3G. is there any problem with giving it like 32G?
not_bob justmessin: The defaults tend to work well for most people. As for more equals better? At some point it's not. And I think the max you can allocate to a jvm is 4 gigs.
justmessin not_bob thanks. just have spare cpu and bw so want to help the network
not_bob Thank you for that.
RN Thanks to the devs for the continuing efforts to unplug the toaster!