SMTD Output Slow/hanging causing error 1003 [SOLUTION: no ::0 listener for ipv6 in RCS]

im_thatoneguy · August 15, 2025, 9:07pm

SMTD is failing to launch most times now due to timeouts. The error code is 1003. The root cause seems to be how long it takes to generate the needed files into the temp folder.

If I run

deadlinecommandbg.exe -outputfiles "B:\Temp\submitOutput.txt" "B:\Temp\submitExitCode.txt" -getrepositorypath submission/3dsmax/Main

it takes on average about 20 -30 seconds to execute every time. This is true of every machine in our network. However, a machine over VPN loads in just 1-2 seconds.

This is also true of the deadlinecommand.exe

im_thatoneguy · August 15, 2025, 9:09pm

It looks like it’s not limited to the get local repository path. Just polling -pools takes 30 seconds for “none” to be returned.

im_thatoneguy · August 15, 2025, 9:30pm

Ok I’ve narrowed it down to an ipv6 issue. Disabling ipv6 on the NIC fixes the problems… Wireshark shows constant issues with Deadline.

ipv4: instant

274	0.150096	192.168.94.32	192.168.94.102	TLSv1.2	499	Application Data
275	0.150729	192.168.94.102	192.168.94.32	TLSv1.2	1514	Application Data, Application Data
276	0.150729	192.168.94.102	192.168.94.32	TLSv1.2	75	Application Data
277	0.150729	192.168.94.102	192.168.94.32	TLSv1.2	88	Application Data
278	0.150775	192.168.94.32	192.168.94.102	TCP	54	39689 → 4433 [ACK] Seq=446 Ack=1516 Win=65535 Len=0
1136	0.425377	192.168.94.32	192.168.94.102	TLSv1.2	499	Application Data
1137	0.426015	192.168.94.102	192.168.94.32	TLSv1.2	1514	Application Data, Application Data
1138	0.426015	192.168.94.102	192.168.94.32	TLSv1.2	109	Application Data, Application Data
1139	0.426051	192.168.94.32	192.168.94.102	TCP	54	39689 → 4433 [ACK] Seq=891 Ack=3031 Win=65535 Len=0
1692	0.695110	192.168.94.32	192.168.94.102	TLSv1.2	499	Application Data
1695	0.695847	192.168.94.102	192.168.94.32	TLSv1.2	1466	Application Data
1696	0.695847	192.168.94.102	192.168.94.32	TLSv1.2	123	Application Data, Application Data
1697	0.695883	192.168.94.32	192.168.94.102	TCP	54	39689 → 4433 [ACK] Seq=1336 Ack=4512 Win=65535 Len=0
2286	0.927390	192.168.94.32	192.168.94.102	TCP	66	43123 → 4433 [SYN] Seq=0 Win=65535 Len=0 MSS=1460 WS=256 SACK_PERM
2287	0.927646	192.168.94.102	192.168.94.32	TCP	66	4433 → 43123 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1460 WS=256 SACK_PERM
2288	0.927761	192.168.94.32	192.168.94.102	TCP	54	43123 → 4433 [ACK] Seq=1 Ack=1 Win=65280 Len=0
2356	0.938819	192.168.94.32	192.168.94.102	TLSv1.2	241	Client Hello (SNI=sfs-services.straightfacehq.com)
2357	0.940240	192.168.94.102	192.168.94.32	TLSv1.2	1401	Server Hello, Certificate, Server Key Exchange, Certificate Request, Server Hello Done
2368	0.949532	192.168.94.32	192.168.94.102	TLSv1.2	1319	Certificate, Client Key Exchange, Certificate Verify, Change Cipher Spec, Encrypted Handshake Message
2427	0.955928	192.168.94.102	192.168.94.32	TLSv1.2	105	Change Cipher Spec, Encrypted Handshake Message
2440	0.965770	192.168.94.32	192.168.94.102	TLSv1.2	499	Application Data
2441	0.966508	192.168.94.102	192.168.94.32	TLSv1.2	1500	Application Data, Application Data
2442	0.966508	192.168.94.102	192.168.94.32	TLSv1.2	123	Application Data, Application Data
2443	0.966546	192.168.94.32	192.168.94.102	TCP	54	39689 → 4433 [ACK] Seq=1781 Ack=6027 Win=65535 Len=0
2460	0.981357	192.168.94.32	192.168.94.102	TLSv1.2	1011	Application Data
2461	0.982327	192.168.94.102	192.168.94.32	TCP	2974	4433 → 43123 [ACK] Seq=1399 Ack=2410 Win=2097920 Len=2920 [TCP PDU reassembled in 2462]
2462	0.982327	192.168.94.102	192.168.94.32	TLSv1.2	330	Application Data
2463	0.982327	192.168.94.102	192.168.94.32	TLSv1.2	124	Application Data, Application Data
2464	0.982383	192.168.94.32	192.168.94.102	TCP	54	43123 → 4433 [ACK] Seq=2410 Ack=4665 Win=65280 Len=0
2782	1.080614	192.168.94.32	192.168.94.102	TLSv1.2	298	Application Data
2783	1.081161	192.168.94.102	192.168.94.32	TLSv1.2	356	Application Data, Application Data
2784	1.081161	192.168.94.102	192.168.94.32	TLSv1.2	88	Application Data
2785	1.081211	192.168.94.32	192.168.94.102	TCP	54	43123 → 4433 [ACK] Seq=2654 Ack=5001 Win=65024 Len=0
3010	1.116287	192.168.94.32	192.168.94.102	TLSv1.2	331	Application Data
3013	1.117522	192.168.94.102	192.168.94.32	TLSv1.2	544	Application Data, Application Data, Application Data, Application Data
3077	1.158155	192.168.94.32	192.168.94.102	TLSv1.2	338	Application Data
3080	1.159750	192.168.94.102	192.168.94.32	TCP	1514	4433 → 43123 [ACK] Seq=5491 Ack=3215 Win=2097152 Len=1460 [TCP PDU reassembled in 3081]
3081	1.159750	192.168.94.102	192.168.94.32	TLSv1.2	323	Application Data
3082	1.159750	192.168.94.102	192.168.94.32	TCP	1514	4433 → 43123 [ACK] Seq=7220 Ack=3215 Win=2097152 Len=1460 [TCP PDU reassembled in 3083]
3083	1.159750	192.168.94.102	192.168.94.32	TLSv1.2	292	Application Data
3084	1.159822	192.168.94.32	192.168.94.102	TCP	54	43123 → 4433 [ACK] Seq=3215 Ack=8918 Win=65280 Len=0
3085	1.159838	192.168.94.102	192.168.94.32	TLSv1.2	1141	Application Data, Application Data, Application Data, Application Data
3133	1.199914	192.168.94.32	192.168.94.102	TCP	54	43123 → 4433 [ACK] Seq=3215 Ack=10005 Win=64256 Len=0
3220	1.231397	192.168.94.32	192.168.94.102	TLSv1.2	312	Application Data
3221	1.231903	192.168.94.102	192.168.94.32	TLSv1.2	434	Application Data, Application Data
3222	1.231903	192.168.94.102	192.168.94.32	TLSv1.2	88	Application Data
3223	1.231948	192.168.94.32	192.168.94.102	TCP	54	43123 → 4433 [ACK] Seq=3473 Ack=10419 Win=65280 Len=0
3264	1.251818	192.168.94.32	192.168.94.102	TLSv1.2	499	Application Data
3265	1.252545	192.168.94.102	192.168.94.32	TLSv1.2	1465	Application Data
3266	1.252545	192.168.94.102	192.168.94.32	TLSv1.2	123	Application Data, Application Data
3267	1.252589	192.168.94.32	192.168.94.102	TCP	54	39689 → 4433 [ACK] Seq=2226 Ack=7507 Win=65535 Len=0
3276	1.264781	192.168.94.32	192.168.94.102	TLSv1.2	317	Application Data
3277	1.265358	192.168.94.102	192.168.94.32	TLSv1.2	500	Application Data, Application Data, Application Data, Application Data
3330	1.297604	192.168.94.32	192.168.94.102	TLSv1.2	362	Application Data
3331	1.298262	192.168.94.102	192.168.94.32	TLSv1.2	309	Application Data, Application Data
3332	1.298262	192.168.94.102	192.168.94.32	TLSv1.2	88	Application Data
3333	1.298299	192.168.94.32	192.168.94.102	TCP	54	43123 → 4433 [ACK] Seq=4044 Ack=11154 Win=64768 Len=0
3440	1.324401	192.168.94.32	192.168.94.102	TLSv1.2	377	Application Data
3441	1.330694	192.168.94.102	192.168.94.32	TLSv1.2	1440	Application Data, Application Data, Application Data
3442	1.330694	192.168.94.102	192.168.94.32	TLSv1.2	88	Application Data
3443	1.330732	192.168.94.32	192.168.94.102	TCP	54	43123 → 4433 [ACK] Seq=4367 Ack=12574 Win=65280 Len=0
3458	1.348847	192.168.94.32	192.168.94.102	TCP	54	43123 → 4433 [FIN, ACK] Seq=4367 Ack=12574 Win=65280 Len=0
3459	1.349001	192.168.94.102	192.168.94.32	TCP	60	4433 → 43123 [ACK] Seq=12574 Ack=4368 Win=2097664 Len=0
3460	1.349122	192.168.94.102	192.168.94.32	TCP	60	4433 → 43123 [FIN, ACK] Seq=12574 Ack=4368 Win=2097664 Len=0
3461	1.349150	192.168.94.32	192.168.94.102	TCP	54	43123 → 4433 [ACK] Seq=4368 Ack=12575 Win=65280 Len=0
3633	1.523618	192.168.94.32	192.168.94.102	TLSv1.2	499	Application Data
3634	1.524413	192.168.94.102	192.168.94.32	TLSv1.2	1514	Application Data, Application Data
3635	1.524413	192.168.94.102	192.168.94.32	TLSv1.2	76	Application Data
3636	1.524413	192.168.94.102	192.168.94.32	TLSv1.2	88	Application Data
3637	1.524454	192.168.94.32	192.168.94.102	TCP	54	39689 → 4433 [ACK] Seq=2671 Ack=9023 Win=65535 Len=0
4020	1.795753	192.168.94.32	192.168.94.102	TLSv1.2	499	Application Data
4021	1.796525	192.168.94.102	192.168.94.32	TLSv1.2	1514	Application Data, Application Data
4022	1.796525	192.168.94.102	192.168.94.32	TLSv1.2	73	Application Data
4023	1.796560	192.168.94.32	192.168.94.102	TCP	54	39689 → 4433 [ACK] Seq=3116 Ack=10502 Win=65535 Len=0
4092	2.072235	192.168.94.32	192.168.94.102	TLSv1.2	499	Application Data
4093	2.072936	192.168.94.102	192.168.94.32	TLSv1.2	1514	Application Data, Application Data
4094	2.072936	192.168.94.102	192.168.94.32	TLSv1.2	108	Application Data, Application Data
4095	2.072972	192.168.94.32	192.168.94.102	TCP	54	39689 → 4433 [ACK] Seq=3561 Ack=12016 Win=65535 Len=0

ipv6

40729	49.975639	2001:428:6402:1094::102	2001:428:6402:1094:3c47:c9f5:9e55:1b8c	TCP	74	4433 → 43532 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0
41362	50.476228	2001:428:6402:1094:3c47:c9f5:9e55:1b8c	2001:428:6402:1094::102	TCP	86	[TCP Port numbers reused] 43532 → 4433 [SYN] Seq=0 Win=65535 Len=0 MSS=1440 WS=256 SACK_PERM
41363	50.476378	2001:428:6402:1094::102	2001:428:6402:1094:3c47:c9f5:9e55:1b8c	TCP	74	4433 → 43532 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0
42150	50.977383	2001:428:6402:1094:3c47:c9f5:9e55:1b8c	2001:428:6402:1094::102	TCP	86	[TCP Port numbers reused] 43532 → 4433 [SYN] Seq=0 Win=65535 Len=0 MSS=1440 WS=256 SACK_PERM
42151	50.977572	2001:428:6402:1094::102	2001:428:6402:1094:3c47:c9f5:9e55:1b8c	TCP	74	4433 → 43532 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0
42985	51.477954	2001:428:6402:1094:3c47:c9f5:9e55:1b8c	2001:428:6402:1094::102	TCP	86	[TCP Port numbers reused] 43532 → 4433 [SYN] Seq=0 Win=65535 Len=0 MSS=1440 WS=256 SACK_PERM
42988	51.478112	2001:428:6402:1094::102	2001:428:6402:1094:3c47:c9f5:9e55:1b8c	TCP	74	4433 → 43532 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0
43663	51.977972	2001:428:6402:1094:3c47:c9f5:9e55:1b8c	2001:428:6402:1094::102	TCP	86	[TCP Port numbers reused] 43532 → 4433 [SYN] Seq=0 Win=65535 Len=0 MSS=1440 WS=256 SACK_PERM
43664	51.978114	2001:428:6402:1094::102	2001:428:6402:1094:3c47:c9f5:9e55:1b8c	TCP	74	4433 → 43532 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0
46240	54.671582	2001:428:6402:1094::102	2001:428:6402:1094:3c47:c9f5:9e55:1b8c	ICMPv6	86	Neighbor Solicitation for 2001:428:6402:1094:3c47:c9f5:9e55:1b8c from 00:15:5d:c9:12:03
46241	54.671637	2001:428:6402:1094:3c47:c9f5:9e55:1b8c	2001:428:6402:1094::102	ICMPv6	86	Neighbor Advertisement 2001:428:6402:1094:3c47:c9f5:9e55:1b8c (sol, ovr) is at 1c:34:da:6f:4c:3b
46242	54.671653	2001:428:6402:1094::102	2001:428:6402:1094:3c47:c9f5:9e55:1b8c	ICMPv6	86	Neighbor Solicitation for 2001:428:6402:1094:3c47:c9f5:9e55:1b8c from 00:15:5d:c9:12:03
46243	54.671664	2001:428:6402:1094:3c47:c9f5:9e55:1b8c	2001:428:6402:1094::102	ICMPv6	86	Neighbor Advertisement 2001:428:6402:1094:3c47:c9f5:9e55:1b8c (sol, ovr) is at 1c:34:da:6f:4c:3b
47928	55.724291	2001:428:6402:1094:3c47:c9f5:9e55:1b8c	2001:428:6402:1094::102	ICMPv6	86	Neighbor Solicitation for 2001:428:6402:1094::102 from 1c:34:da:6f:4c:3b
47929	55.724450	2001:428:6402:1094::102	2001:428:6402:1094:3c47:c9f5:9e55:1b8c	ICMPv6	86	Neighbor Advertisement 2001:428:6402:1094::102 (sol, ovr) is at 00:15:5d:c9:12:03
47930	55.724450	2001:428:6402:1094::102	2001:428:6402:1094:3c47:c9f5:9e55:1b8c	ICMPv6	86	Neighbor Advertisement 2001:428:6402:1094::102 (sol, ovr) is at 00:15:5d:c9:12:03
47931	55.724518	2001:428:6402:1094::102	2001:428:6402:1094:3c47:c9f5:9e55:1b8c	ICMPv6	86	Neighbor Advertisement 2001:428:6402:1094::102 (sol, ovr) is at 00:15:5d:c9:12:03
47932	55.724558	2001:428:6402:1094::102	2001:428:6402:1094:3c47:c9f5:9e55:1b8c	ICMPv6	86	Neighbor Advertisement 2001:428:6402:1094::102 (sol, ovr) is at 00:15:5d:c9:12:03

im_thatoneguy · August 15, 2025, 9:43pm

Ok, I’ve figured out what’s happening at least when I look at both ipv6 and ipv4 simultaneously.

The ipv6 implementation attempts to reach the host over ipv6. It gets a bunch of RESET responses and errors over ports already bound.
It then neighbor solicits the Deadline Repository MAC/IP from my NIC.
The Deadline Repository responds.
The Deadline Repository neighbor solicits me
My NIC responds.
Then it just sits there and nothing happens for about 20 seconds (presumably a timeout in DeadlineCommand)
then it tries over ipv4 and instantly succeeded.

im_thatoneguy · August 15, 2025, 10:12pm

Hack workaround on Deadline RCS machine.

netsh advfirewall firewall add rule name="Deadline RCS v6 inbound" dir=in action=allow protocol=TCP localport=4433

netsh interface portproxy add v6tov4 listenaddress=:: listenport=4433 connectaddress=127.0.0.1 connectport=4433

netsh interface portproxy show v6tov4
netstat -ano -p tcpv6 | findstr :4433

TCP [::]:4433 [::]:0 LISTENING 2748