C#•2y ago

❔ Why could NuGet restore randomly timeout inside GitHub Actions?

So frequently nuget restore timeout inside GitHub Actions for months and I can't figure out what could be the reason for this. Is it could be because multiple GitHub Actions jobs trying to restore in same time and NuGet servers thinking it is spam and blocking it? build.yml: https://github.com/ShareX/ShareX/blob/develop/.github/workflows/build.yml#L72 failed workflow: https://github.com/ShareX/ShareX/actions/runs/4933908932/jobs/8818333221 Also I'm open to advices about how to retry GitHub Actions job when it fails once.

96 Replies

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

problem is because it happens randomly, even if i enable verbose logging, unfortunately we won't be able to reproduce the error immediately

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

but i have feeling that nuget servers just don't respond at all and then it timeouts

JaexOP•2y ago

https://jaex.getsharex.com/2023/05/chrome_qYBNW5UqVs.png

JaexOP•2y ago

hm which one i should select gonna try -v d

Unknown User•2y ago

Message Not Public

gerard•2y ago

Maybe you could cache the packages? Sadly there isn't docs for NuGet: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows

GitHub Docs

Caching dependencies to speed up workflows - GitHub Docs

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

https://jaex.getsharex.com/2023/05/chrome_lWLm0uvI7U.png

JaexOP•2y ago

normally it takes around 1 minute to restore so I still have doubts about it timeouts because it takes too long to restore people restore very huge files which would take long time i still think it timeout because 5 jobs in same time trying to restore and nuget servers spam protection stops responding to one of them then it timeout after that

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

i will ofc add logs but we won't be able to get results for some time 😄 i will let you know when timeout happens again we didn't had to wait too long it happened in first commit lol

JaexOP•2y ago

https://github.com/ShareX/ShareX/actions/runs/4935933961/jobs/8822855210

GitHub

Added -Verbosity detailed · ShareX/ShareX@9873022

ShareX is a free and open source program that lets you capture or record any area of your screen and share it with a single press of a key. It also allows uploading images, text or other types of f...

JaexOP•2y ago

NuGet.Commands.CommandException: MsBuild timeout out while trying to get project to project references.

time to Google it

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

i still cannot find solution on Google

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

weirdly it is happens so frequently

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

i think it started to happen after i added 5. job thats why i felt like it is spam protection all 5 jobs could be sharing same ip and when all do restore in same time nuget servers could be thinking it is spam

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

yea i wonder can i find someone from nuget team in this server to ask 😂

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

i dont think so

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

i did not manually create lockfile and dunno what it is

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

btw i was thinking what if i use "powershell try catch" to try restore again, would it work?

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

doing one more restore at catch i mean

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

but i would prefer to solve the main issue because this delays builds 2 minutes+

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

-Project2ProjectTimeOut

Timeout in seconds for resolving project-to-project references.

-Project2ProjectTimeOut

Timeout in seconds for resolving project-to-project references.

I can try increasing this and see is it still timeout

nuget restore "${{ env.SOLUTION_FILE_PATH }}" -Project2ProjectTimeOut 300000 -Verbosity detailed

nuget restore "${{ env.SOLUTION_FILE_PATH }}" -Project2ProjectTimeOut 300000 -Verbosity detailed

5 mins timeout

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

waait is it seconds i used ms

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

MSBuild P2P timeout [ms]: 300000000 lol

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

yea logs mislead me

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

i dont think it can use same location on disk but how ip can be same is i dunno

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

one job currently stuck in nuget restore btw

JaexOP•2y ago

https://jaex.getsharex.com/2023/05/chrome_PBaKjUmwJI.png

JaexOP•2y ago

why github actions logs don't have timestamp blobthinking

had them when you hovered line number

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

https://jaex.getsharex.com/2023/05/chrome_onATebmKq5.png

JaexOP•2y ago

jesus it took 5 minutes to complete

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

so my theory of spam protection is invalidated

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

https://github.com/ShareX/ShareX/actions/runs/4936117716/jobs/8823266606

GitHub

Added -Project2ProjectTimeOut 300000 · ShareX/ShareX@3b88f13

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

you can see the full logs right? when i open the page in browser incognito mode i cant see the logs

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

but probably requires login

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

nuget docs said detailed is max for nuget restore dotnet restore had higher level but dunno would that arg work for nuget restore

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

i saw it right before see your ss 😂 i assumed it would be available when hovered with cursor like in appveyor

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

github machines rubbish

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

or nuget servers rubbish difficult to know

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

it is pretty small

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

i dont even know what that command does

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

https://learn.microsoft.com/en-us/nuget/reference/cli-reference/cli-ref-restore

NuGet CLI restore command

Reference for the nuget.exe restore command

JaexOP•2y ago

-Verbosity [normal|quiet|detailed] are you sure thats usable in nuget cli because what you are looking at is dotnet restore cli

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

other job: https://jaex.getsharex.com/2023/05/chrome_girSCudigO.png

JaexOP•2y ago

vs https://jaex.getsharex.com/2023/05/chrome_h78zMVQbqA.png

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

thank you

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

you helped to diagnose a lot 😂 i was more clueless before now i'm closer

Unknown User•2y ago

Message Not Public

gerard•2y ago

Maybe the problem is signature verification? It seems like it's a Windows thing: https://github.com/NuGet/Home/issues/11548 You can disable it with a env-variable:

Important: setting NUGET_CERT_REVOCATION_MODE to offline indeed improves restore times substantially (down to 1m 22s in this case). https://github.com/StackExchange/StackExchange.Redis/pull/1973/commits/275f5756cbc526df7e3bff8a82dac6dd52f2186d

GitHub

[Bug]: Slow Restores on Windows in GitHub Actions - Signature verif...

NuGet Product Used dotnet.exe Product Version 6.0.101 Repro Steps & Context Running an identical restore in GitHub Actions, times are dramatically higher on Windows agents (2019 and 2022 tested...

gerard•2y ago

If it's still slow, maybe you can try Windows Server 2019 (windows-2019) because windows-2022 used to be much slower. I don't know if this is still the case: https://github.com/actions/runner-images/issues/5166

GitHub

Windows Server 2022 builds are taking 4x longer than on Windows Ser...

Description Over the last 2 weeks I've noticed runs on the windows-2022 image taking 4x longer, and sometimes more than that, than on the windows-2019 image. In all of these instances nothing&#...

Accord•2y ago

Was this issue resolved? If so, run /close - otherwise I will mark this as stale and this post will be archived until there is new activity.

JaexOP•2y ago

https://github.com/actions/cache/blob/main/examples.md#c---nuget I'm checking this now created empty packages.lock.json files which populated on first compile i wish i could use it in parent solution folder tho instead of each project folder

JaexOP•2y ago

https://jaex.getsharex.com/2023/05/GitHubDesktop_jp8YwSfKc6.png

JaexOP•2y ago

https://github.com/ShareX/ShareX/actions/caches they did not put download button so can't verify cache content notlikeblob

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

https://jaex.getsharex.com/2023/05/chrome_x5HxgBE8nl.png

JaexOP•2y ago

am i need to use another parameter for nuget to restore from cache https://github.com/ShareX/ShareX/blob/develop/.github/workflows/build.yml#L70

- name: NuGet Restore
  if: steps.cache.outputs.cache-hit != 'true'
  run: nuget restore MyProject.sln

- name: NuGet Restore
  if: steps.cache.outputs.cache-hit != 'true'
  run: nuget restore MyProject.sln

let me try this

JaexOP•2y ago

didn't work 😦 https://jaex.getsharex.com/2023/05/chrome_zSWWHXCJbG.png

JaexOP•2y ago

@TeBeClone did you cache nuget packages before?

Unknown User•2y ago

Message Not Public

JaexOP•2y ago

chatgpt couldn't help too 😂 https://jaex.getsharex.com/2023/05/61KmjXWflu.png

JaexOP•2y ago

it shows what i already did i wonder is this path wrong for Windows: path: ~/.nuget/packages but if it were wrong then cache would be empty https://github.com/ShareX/ShareX/actions/caches 34 MB there i have suspicion now that nuget actually using cache correctly but still "GenerateRestoreGraphFile" part takes all that time

Accord•2y ago

Was this issue resolved? If so, run /close - otherwise I will mark this as stale and this post will be archived until there is new activity.

Gaming

Programming

❔ Why could NuGet restore randomly timeout inside GitHub Actions?