climateprediction.net (CPDN) home page
Thread 'Uploads not working'

Thread 'Uploads not working'

Message boards : Number crunching : Uploads not working
Message board moderation

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
Dark Angel

Send message
Joined: 31 May 18
Posts: 53
Credit: 4,725,987
RAC: 9,174
Message 70473 - Posted: 20 Feb 2024, 11:32:09 UTC - in response to Message 70472.  

I changed my VM networking from Bridged to NAT to see if it makes any difference but I'll have to wait until the server unlocks the files again.
I doubt that'll make any difference as that just affects how the VM appears on the local network. The outside world only sees your router IP address, not the local IPs.


I figured that, but thought it was worth a try. It just made my proxy logs harder to read.
ID: 70473 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1049
Credit: 16,432,494
RAC: 17,331
Message 70474 - Posted: 20 Feb 2024, 11:56:16 UTC - in response to Message 70469.  

Second attempt:
Tracing route to upload7.cpdn.org [141.223.16.156]
over a maximum of 30 hops:
...<snipped>...
 14   225 ms   224 ms   228 ms  112.174.48.1
 15     *        *        *     Request timed out.
 16   229 ms   229 ms   229 ms  210.223.242.98
 17   230 ms   230 ms   229 ms  141.223.180.2
 18   232 ms   227 ms   226 ms  141.223.253.61
 19   231 ms   231 ms   231 ms  141.223.16.93
 20   231 ms   230 ms   229 ms  eawah.postech.ac.kr [141.223.16.156]
With traceroute from my machine to upload7 (aka postech.ac.kr : 141.223.16.256) I can't get to upload7 at the moment, but I do have a similar route into S.Korea:
13  112.174.91.173 (112.174.91.173)  279.613 ms 112.174.86.189 (112.174.86.189)  276.004 ms 112.174.91.213 (112.174.91.213)  275.984 ms
14  112.190.29.250 (112.190.29.250)  277.345 ms 112.174.48.105 (112.174.48.105)  280.747 ms  279.017 ms
15  * * *
16  210.223.242.98 (210.223.242.98)  276.224 ms  277.691 ms  283.102 ms
IP lookup gives 112.174.91.173 which is in Seoul itself, but after that it jumps around, to: 112.190.29.250 in Seongnam, 112.174.48.105 in Gwangju, back to Seongnam (210.223.242.98), before my last reply is 141.223.253.60 in Pohang way south-east of Seoul, where Postech Uni is located. At a guess I'd say there's congestion on the route through Korea to the university?
---
CPDN Visiting Scientist
ID: 70474 · Report as offensive     Reply Quote
Dark Angel

Send message
Joined: 31 May 18
Posts: 53
Credit: 4,725,987
RAC: 9,174
Message 70475 - Posted: 20 Feb 2024, 11:59:31 UTC

Ok, I did a thing. It appears to have worked. In a sense I was right about not having this problem if they were linux tasks but only because if they were I wouldn't be using a VM to run them. The problem is a bug in the Virtualbox Bridged Network adaptor.
The change I made wasn't NAT, it was to the Windows network device driver settings.
Specifically I disabled “Large Send Offload (IPv4)” in the adaptor settings.
BOOM! Next upload file runs at 2000KBps instead of 3.2

I found the solution here in the last post: https://forums.virtualbox.org/viewtopic.php?t=110486
ID: 70475 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1049
Credit: 16,432,494
RAC: 17,331
Message 70476 - Posted: 20 Feb 2024, 12:29:04 UTC - in response to Message 70475.  

Thanks for posting that. I'm also using VMs with bridged adaptors. I will check it!
ID: 70476 · Report as offensive     Reply Quote
Glenn Carver

Send message
Joined: 29 Oct 17
Posts: 1049
Credit: 16,432,494
RAC: 17,331
Message 70477 - Posted: 20 Feb 2024, 12:46:35 UTC - in response to Message 70475.  

Which network adapter are you using with VirtualBox out of interest? eno1 or wlp6s0? I checked my Win10 VM and I have wlp6s0 enabled (aka wireless), which strictly speaking I shouldn't have for bridging but I was lazy and selected the first option. Curious to know if the problem only affects the ethernet adapter, eno1.

I was also reading this article about problems with the default VirtualBox ethernet network adapter being the root cause:
https://petri.com/how-to-improve-network-performance-in-windows-virtualbox-guests/
It suggests installing the virtio-net adapter type for better performance. I was trying to check the driver was not junk when I saw your fix. I might still try this though.

Ok, I did a thing. It appears to have worked. In a sense I was right about not having this problem if they were linux tasks but only because if they were I wouldn't be using a VM to run them. The problem is a bug in the Virtualbox Bridged Network adaptor.
The change I made wasn't NAT, it was to the Windows network device driver settings.
Specifically I disabled “Large Send Offload (IPv4)” in the adaptor settings.
BOOM! Next upload file runs at 2000KBps instead of 3.2

I found the solution here in the last post: https://forums.virtualbox.org/viewtopic.php?t=110486

---
CPDN Visiting Scientist
ID: 70477 · Report as offensive     Reply Quote
Dark Angel

Send message
Joined: 31 May 18
Posts: 53
Credit: 4,725,987
RAC: 9,174
Message 70485 - Posted: 20 Feb 2024, 20:06:16 UTC - in response to Message 70477.  

Which network adapter are you using with VirtualBox out of interest? eno1 or wlp6s0? I checked my Win10 VM and I have wlp6s0 enabled (aka wireless), which strictly speaking I shouldn't have for bridging but I was lazy and selected the first option. Curious to know if the problem only affects the ethernet adapter, eno1.

I was also reading this article about problems with the default VirtualBox ethernet network adapter being the root cause:
https://petri.com/how-to-improve-network-performance-in-windows-virtualbox-guests/
It suggests installing the virtio-net adapter type for better performance. I was trying to check the driver was not junk when I saw your fix. I might still try this though.

Ok, I did a thing. It appears to have worked. In a sense I was right about not having this problem if they were linux tasks but only because if they were I wouldn't be using a VM to run them. The problem is a bug in the Virtualbox Bridged Network adaptor.
The change I made wasn't NAT, it was to the Windows network device driver settings.
Specifically I disabled “Large Send Offload (IPv4)” in the adaptor settings.
BOOM! Next upload file runs at 2000KBps instead of 3.2

I found the solution here in the last post: https://forums.virtualbox.org/viewtopic.php?t=110486


I have different adaptors to you, I think it's host specific. This is the selection I have:
(can't attach a picture so I'll have to type it out)
PCnet-PCI II(Am79C970A)
PCnet-FAST III (Am79C973)
Intel PRO/1000 MT Desktop (82540EM)
Intel PRO/1000 T Server (82543GC)
Intel PRO/1000 MT Server (82545EM)
Paravirtualised Network (virtio-net)

I found my VM refused to connect to the network with either of the PCnet adaptors or the Paravirtualised adaptor and had the same issue with all three Intel adaptors.
The adaptor name in my case is enp7s0 (Ethernet) with the options of wlp6s0 (WiFi) or ham0 (I run a VPN to my home server for my mobile devices)

I saw one upload go at over 3000KBps just now. That's rather a spectacular improvement over 3.2KBps for what amounts to a very simple change.[/img]
ID: 70485 · Report as offensive     Reply Quote
ProfileDave Jackson
Volunteer moderator

Send message
Joined: 15 May 09
Posts: 4540
Credit: 19,024,725
RAC: 20,592
Message 70486 - Posted: 20 Feb 2024, 20:21:17 UTC

Interesting. I didn't think I had a problem despite using defaults with Win10 under VB but when one of my tasks finished just after six, the out.zip and 24.zip were only managing about 6KB/s between them. However once the relatively small out.zip finished, the 24.zip got up to over 25KB/s before I left to answer the call of dinner! It had cleared by the time I returned. Other uploads that I have monitored have all gone through normally for my bored band which maxes out at 100KB/s anyway. Not sure if the slow upload was connected with this issue or not?
ID: 70486 · Report as offensive     Reply Quote
Dark Angel

Send message
Joined: 31 May 18
Posts: 53
Credit: 4,725,987
RAC: 9,174
Message 70488 - Posted: 20 Feb 2024, 21:44:04 UTC

It's a simple and reversible change if you want to try it.

Network and Internet > Change Adaptor Settings > (select adaptor, I double clicked) > Properties > Configure > Advanced Tab > Large Send Offload (IPv4) > set to disabled > ok and close back out.

There's probably a quicker way to that menu but that got me there.
ID: 70488 · Report as offensive     Reply Quote
Previous · 1 · 2

Message boards : Number crunching : Uploads not working

©2024 cpdn.org