Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NEXT-1143] Dev mode slow compilation #48748

Open
1 task done
jeengbe opened this issue Apr 23, 2023 · 516 comments
Open
1 task done

[NEXT-1143] Dev mode slow compilation #48748

jeengbe opened this issue Apr 23, 2023 · 516 comments
Assignees
Labels
linear: next Confirmed issue that is tracked by the Next.js team.

Comments

@jeengbe
Copy link

jeengbe commented Apr 23, 2023

⚠️ this original post has been edited by @timneutkens to reflect this comment ⚠️

Changes in the past week

I've been investigating this over the past week. Made a bunch of changes, some make a small impact, some make a large impact. Here's a list:

You can try them using npm install next@canary.

Help Investigate

In order to help me investigate this I'll ideally need an application that can be run, if you can't provide that (I understand if you can't) please provide the .next/trace file.

If possible follow these steps which would give me the best picture to investigate:

  • npm install next@canary (use the package manager you're using) -- We want to make sure you're using the very latest version of Next.js which includes the fixes mentioned earlier.
  • rm -rf .next
  • start development using the NEXT_CPU_PROF=1 and NEXT_TURBOPACK_TRACING=1 (regardless of if you're using Turbopack, it only affects when you do use Turbopack) environment variable. E.g.:
    • npm: NEXT_TURBOPACK_TRACING=1 NEXT_CPU_PROF=1 npm run dev
    • yarn: NEXT_TURBOPACK_TRACING=1 NEXT_CPU_PROF=1 yarn dev
    • pnpm: NEXT_TURBOPACK_TRACING=1 NEXT_CPU_PROF=1 pnpm dev
  • Wait a few seconds
  • Open a page that you're working on
  • Wait till it's fully loaded
  • Wait a few seconds
  • Make an edit to a file that holds a component that is on the page
  • Wait for the edit to apply
  • Wait a few seconds
  • Make another edit to the same file
  • Wait a few seconds
  • Exit the dev command (ctrl+c)
  • Upload the CPU traces put in the root of the application directory to https://gist.github.com
  • Upload the .next/trace file to https://gist.github.com -- Please don't run trace-to-tree yourself, as I use some other tools (e.g. Jaeger) that require the actual trace file.
  • If you're using Turbopack upload the .next/trace.log as well, if it's too large for GitHub gists you can upload it to Google Drive or Dropbox and share it through that.
  • Upload next.config.js (if you have one) to https://gist.github.com
  • Share it here

Known application-side slowdowns

To collect things I've seen before that cause slow compilation as this is often the root cause:

  • If you're on Windows, disable Windows Defender, it's a known cause of extreme slowdowns in filesystem access as it sends each file to an external endpoint before allowing to read/write
  • Filesystem slowness overall is what we've seen as the cause of problems, e.g. with Docker
  • react-icons, material icons, etc. Most of these libraries publish barrel files with a lot of re-exports. E.g. material-ui/icons ships 5500 module re-exports, which causes all of them to be compiled. You have to add modularizeImports to reduce it, here's an example: long compile times locally - along with "JavaScript heap out of memory" since upgrade to NextJS 13 #45529 (comment)
  • Custom postcss config, e.g. tailwindcss with a content setting that tries to read too many files (e.g. files not relevant for the application)

This and other slowdown reports are currently the top priority for our team. We'll continue optimizing Next.js with webpack where possible.
The Turbopack team is currently working on getting all Next.js integration tests passing when using Turbopack as we continue working towards stability of Turbopack.

Original post

Verify canary release

  • I verified that the issue exists in the latest Next.js canary release

Provide environment information

Operating System:
      Platform: linux
      Arch: x64
      Version: #1 SMP Fri Jan 27 02:56:13 UTC 2023
    Binaries:
      Node: 18.13.0
      npm: 8.19.3
      Yarn: 1.22.18
      pnpm: 7.30.5
    Relevant packages:
      next: 13.3.1
      eslint-config-next: 13.3.1
      react: 18.2.0
      react-dom: 18.2.0

Which area(s) of Next.js are affected? (leave empty if unsure)

No response

Link to the code that reproduces this issue

https://github.com/DigitalerSchulhof/digitaler-schulhof

To Reproduce

Note that I have been unable to replicate this issue in a demo repository.

Describe the Bug

The issue is that Next.js is generally slow in dev mode. Navigating to new pages takes several seconds:

[next] ready - started server on 0.0.0.0:3000, url: http://localhost:3000
[next] info  - Loaded env from /home/jeengbe/dsh/digitaler-schulhof/.env
[next] warn  - You have enabled experimental feature (appDir) in next.config.js.
[next] warn  - Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Use at your own risk.
[next] info  - Thank you for testing `appDir` please leave your feedback at https://nextjs.link/app-feedback
[next] event - compiled client and server successfully in 1574 ms (267 modules)
[next] wait  - compiling...
[next] event - compiled client and server successfully in 219 ms (267 modules)
[next] wait  - compiling /(schulhof)/Schulhof/page (client and server)...
[next] event - compiled client and server successfully in 3.6s (1364 modules)
[next] wait  - compiling /(schulhof)/Schulhof/(login)/Anmeldung/page (client and server)...
[next] event - compiled client and server successfully in 1920 ms (1411 modules)
[next] wait  - compiling /api/schulhof/auth/login/route (client and server)...
[next] event - compiled client and server successfully in 625 ms (1473 modules)
[next] wait  - compiling /(schulhof)/Schulhof/Nutzerkonto/page (client and server)...
[next] event - compiled client and server successfully in 1062 ms (1482 modules)
[next] wait  - compiling /(schulhof)/Schulhof/Nutzerkonto/Profil/page (client and server)...
[next] event - compiled client and server successfully in 1476 ms (1546 modules)
[next] wait  - compiling /(schulhof)/Schulhof/Nutzerkonto/Profil/Einstellungen/page (client and server)...
[next] event - compiled client and server successfully in 2.1s (1559 modules)

The only somewhat reasonable time would be 600ms for the API route /api/schulhof/auth/login/route, even though that is still quite a lot slower than what it should be given its size.

It also doesn't look right to compile ~1500 modules for each page, as most of them should be cached. The pages are not very different.

Even an empty API route takes several hundreds of ms. The following example contains solely type exports:

[next] wait  - compiling /api/schulhof/administration/persons/persons/settings/route (client and server)...
[next] event - compiled successfully in 303 ms (107 modules)

I am not exactly sure how to read trace trees, but what stands out is that there are (over multiple runs) several entry next-app-loader that take 2+ seconds to complete:

│  │  ├─ entry next-app-loader?name=app/(schulhof)/Schulhof/page&page=/(schulhof)/Schulhof/page&appPaths=/(schulhof)/Schulhof/page&pagePath=private-next-app-dir/(schulhof)/Schulhof/page.tsx&appDir=/home/jeengbe/dsh/digitaler-schulhof/app&pageExtensions=tsx&pageExtensions=ts&pageExtensions=jsx&pageExtensions=js&rootDir=/home/jeengbe/dsh/digitaler-schulhof&isDev=true&tsconfigPath=tsconfig.json&assetPrefix=&nextConfigOutput=! 1.9s

Find both dev and build traces here: https://gist.github.com/jeengbe/46220a09846de6535c188e78fb6da03e

Note that I have modified trace-to-tree.mjs to include event times for all events.

It also seems unusual that none of the modules have child traces.

Expected Behavior

Initial load and navigating should be substantially faster.

Which browser are you using? (if relevant)

No response

How are you deploying your application? (if relevant)

No response

From SyncLinear.com | NEXT-1143

@jeengbe jeengbe added the bug Issue was opened via the bug report template. label Apr 23, 2023
@joacub
Copy link

joacub commented Apr 24, 2023

same here, and in docker env is even worse, seems like is processing same files over and over without caching them.

@jinojacob15
Copy link

Same for me also dev env ,navigating to different pages via link component is pretty slow

@denu5
Copy link

denu5 commented Apr 25, 2023

+1 its same here, hitting the page first time seems fine but routing via links gets stuck

@joacub
Copy link

joacub commented Apr 25, 2023

last canary version has a better cold build times improvements, still slow like 2-5 seconds (in docker env) waiting but much better

the version im talking about is 13.3.2-canary.6

@denu5
Copy link

denu5 commented May 2, 2023

Hey, @jeengbe there have been some patch updates (13.3.1 -> 13.3.4) did it improve for you?

@jeengbe
Copy link
Author

jeengbe commented May 2, 2023

Hi @denu5,

unfortunately, I can't report any real performance changes since I opened this issue.

You might want to check out the above linked issue in the TypeScript repo though - might be related.

@joacub
Copy link

joacub commented May 2, 2023

As @jeengbe mention there is no performance improvement, there is also a lot of I/O I don’t know why, one request gets pretty much like 1gb-2gb of io. And it is very slow.

@jeengbe
Copy link
Author

jeengbe commented May 2, 2023

As @jeengbe mention there is no performance improvement, there is also a lot of I/O I don’t know why, one request gets pretty much like 1gb-2gb of io. And it is very slow.

Unfortunately, I can't confirm this for my case

image

@joacub
Copy link

joacub commented May 2, 2023

As @jeengbe mention there is no performance improvement, there is also a lot of I/O I don’t know why, one request gets pretty much like 1gb-2gb of io. And it is very slow.

Unfortunately, I can't confirm this for my case

image

That’s pretty good, in my case there is a lot of I/O, maybe is because I’m using material-ui but I think is too much even though.

@jeengbe
Copy link
Author

jeengbe commented May 2, 2023

As @jeengbe mention there is no performance improvement, there is also a lot of I/O I don’t know why, one request gets pretty much like 1gb-2gb of io. And it is very slow.

Unfortunately, I can't confirm this for my case
image

That’s pretty good, in my case there is a lot of I/O, maybe is because I’m using material-ui but I think is too much even though.

Possibly, it would align with what your trace shows: #48407 (comment)

@langfordG
Copy link

langfordG commented May 4, 2023

I see that slow route changes in dev mode are showing a '[Fast Refresh] rebuilding' message in the browser console. Sometimes it performs a full page reload when changing routes even if no files have been edited.

@timneutkens timneutkens added the linear: next Confirmed issue that is tracked by the Next.js team. label May 10, 2023
@timneutkens timneutkens changed the title Dev mode very slow navigation (Slow entry next-app-loader spans?) [NEXT-1143] Dev mode very slow navigation (Slow entry next-app-loader spans?) May 10, 2023
@AsathalMannan
Copy link

its slowing down the development..!

@vajajak
Copy link

vajajak commented May 21, 2023

Having the same issue here, in the Docker environment it's come to a point where it's almost unusable, and sometimes I even have to do a hard reload, after waiting too long for navigation. This is the case both with component from next/navigation, as with the router.push (useRouter hook imported from next/navigation). We're using Next.js 13.4.2.

@joacub
Copy link

joacub commented May 22, 2023

Having the same issue here, in the Docker environment it's come to a point where it's almost unusable, and sometimes I even have to do a hard reload, after waiting too long for navigation. This is the case both with component from next/navigation, as with the router.push (useRouter hook imported from next/navigation). We're using Next.js 13.4.2.

same here, it is almost not usable in dokcer enviorements, but also outside docker is very slow, something is not working nice. this is painfully slow.

@JoshApp
Copy link

JoshApp commented May 27, 2023

Yeah same for me. I used to remote develop inside our k8s cluster but dev --turbo is super slow inside a container and causes my health check endpoint to sigkill it regularly.

The whole app router is super slow when containerized in Dev mode.

It works perfectly fine when I run both on my local machine and connect it via reverse proxy. This way it's faster than the old setup (which was not significantly faster before) and takes advantage of preloading pages via next/link. I see inconsistencies in caching too where it's a mix of instant navigation or long builds (around 3.5k modules for some things) around 2-10 sec.

Also there is this weird thing happening that a page compiles just fine and then later it grinds to a halt being stuck in waiting for compiling forever until the pod is crashed.

@Rykuno
Copy link

Rykuno commented May 27, 2023

I love next, but this is a complete show stopper. Sometimes it takes 10+ seconds outside of docker for me on a Mac M2 to navigate one page.

This is insane.

@joacub
Copy link

joacub commented May 27, 2023

I love next, but this is a complete show stopper. Sometimes it takes 10+ seconds outside of docker for me on a Mac M2 to navigate one page.

This is insane.

Yep even more I get sometimes 50 seconds in a simple page, that’s because is also building other things related to that in pralllel I guess.

not even outside docker, i just make a test to work outside docker and timing is exactly the same no difference…. Is getting slower and slower

@joacub
Copy link

joacub commented May 27, 2023

Btw webpack lazy building cold is faster than turbopack 🙂 by far

@Rykuno
Copy link

Rykuno commented May 27, 2023

Btw webpack lazy building cold is faster than turbopack 🙂 by far

Yes! I'm surprised this is not more prevalent as an issue atm; unless turbo will somehow fix all of this in 13.5 and they're waiting to address it.

What configs do you have for the faster webpack builds? I've tried quite a bit and can't lower my built time by much. I need a temporary fix for this ASAP :(

@oalexdoda
Copy link

A month later no updates on this? Makes development on appDir absolutely impossible. @timneutkens ?

Linked a bunch of related issues on this:
#50332

@JunkyDeLuxe
Copy link

I confirm that next app dir on dev mode and dynamic routing are very very slow on docker now

@timneutkens
Copy link
Member

timneutkens commented Jun 6, 2023

Changes in the past week

I've been investigating this over the past week. Made a bunch of changes, some make a small impact, some make a large impact. Here's a list:

You can try them using npm install next@canary.

Help Investigate

In order to help me investigate this I'll ideally need an application that can be run, if you can't provide that (I understand if you can't) please provide the .next/trace file.

If possible follow these steps which would give me the best picture to investigate:

  • npm install next@canary (use the package manager you're using) -- We want to make sure you're using the very latest version of Next.js which includes the fixes mentioned earlier.
  • rm -rf .next
  • start development using the NEXT_CPU_PROF=1 environment variable. E.g.:
    • npm: NEXT_CPU_PROF=1 npm run dev
    • yarn: NEXT_CPU_PROF=1 yarn dev
    • pnpm: NEXT_CPU_PROF=1 pnpm dev
  • Wait a few seconds
  • Open a page that you're working on
  • Wait till it's fully loaded
  • Wait a few seconds
  • Make an edit to a file that holds a component that is on the page
  • Wait for the edit to apply
  • Wait a few seconds
  • Make another edit to the same file
  • Wait a few seconds
  • Exit the dev command (ctrl+c)
  • Upload the CPU traces put in the root of the application directory to https://gist.github.com
  • Upload the .next/trace file to https://gist.github.com -- Please don't run trace-to-tree yourself, as I use some other tools (e.g. Jaeger) that require the actual trace file.
  • Share it here

Known application-side slowdowns

To collect things I've seen before that cause slow compilation as this is often the root cause:

  • If you're on Windows, disable Windows Defender, it's a known cause of extreme slowdowns in filesystem access as it sends each file to an external endpoint before allowing to read/write
  • Filesystem slowness overall is what we've seen as the cause of problems, e.g. with Docker
  • react-icons, material icons, etc. Most of these libraries publish barrel files with a lot of re-exports. E.g. material-ui/icons ships 5500 module re-exports, which causes all of them to be compiled. You have to add modularizeImports to reduce it, here's an example: long compile times locally - along with "JavaScript heap out of memory" since upgrade to NextJS 13 #45529 (comment)
  • Custom postcss config, e.g. tailwindcss with a content setting that tries to read too many files (e.g. files not relevant for the application)

This and other slowdown reports are currently the top priority for our team. We'll continue optimizing Next.js with webpack where possible.
The Turbopack team is currently working on getting all Next.js integration tests passing when using Turbopack as we continue working towards stability of Turbopack.

@timneutkens timneutkens changed the title [NEXT-1143] Dev mode very slow navigation (Slow entry next-app-loader spans?) [NEXT-1143] Dev mode slow navigation Jun 6, 2023
@timneutkens timneutkens changed the title [NEXT-1143] Dev mode slow navigation [NEXT-1143] Dev mode slow compilation Jun 6, 2023
@timneutkens
Copy link
Member

Changed the initial post in this issue to reflect my reply above in order to ensure people see it as the first thing when opening the issue. I'm going to close the duplicate issues reporting similar slowdowns in favor of this one.

I'll need help from you all to ensure this thread doesn't spiral in "It is slow" comments that are not actionable when e.g. without traces / reproduction / further information. Thank you 🙏

@wadamek65
Copy link

@timneutkens Thanks for taking a look! I think you're right; I think the slowness comes from the Vanilla Extract plugin, probably in combination with Mantine that we're using. I've seen quite a few issues regarding this in their repository. I tried adding optimizePackageImports: ['@mantine/core', '@mantine/dropzone', '@mantine/tiptap'], but it didn't seem to help a lot. But at this point, it seems more like a webpack issue to debug rather than a NextJS one, so I suppose I'll be taking a look into that next.

Thanks for taking the time to look into this. I really appreciate it! By the way, is there an instruction somewhere on how to load those traces into Jaeger and debug by ourselves?

@dstoyanoff
Copy link

@dstoyanoff I'll have a look at the trace soon, you should be able to try Turbopack with this configuration based on that being the only webpack configuration: https://x.com/timneutkens/status/1805180098703675882

Hello @timneutkens, were you able to look into this?

@rzzapp
Copy link

rzzapp commented Aug 7, 2024

@timneutkens thanks really for good inside, we did actually able to drop our compile/load time from 100-200s to around 1-2s. Main issues we missed:

  • content directory for tailwind.config.js which we just ignored node_modules check for one package (which we did it for 4 packages but 1 of them was missing it)
  • Barrel files within monorepo packages which were preventing tree-shaking, then added it to optimizePackageImports

Also we upgrade from v14.1.4 to v14.2.5.

So far cold start takes around 10-20s but later gets much much faster.

@kl-thamm
Copy link

kl-thamm commented Aug 7, 2024

@timneutkens Can we get a bit more systematic here and maybe also collect more general instructions? I know the detailed debugs are helpful for individual users and also for next to understand what is going on, but it will not allow for a good dx out of the box. Many recommendations you need to find by browsing through very large GitHub issues and lots of it is outdated.

In addition to that, there still seem to be a more fundamental issues, because I keep reading here about initial compile times of 10s and even reloads that exceed seconds after a very fair share of optimization. I can only compare that to other frameworks I worked with and given the very high hardware demands (e.g. 64GB RAM and M1 Max are not sufficient to be really fast), this is frustrating.

So what I gathered so far:

What When How Documentation Remark
Upgrade Next Always Depending on version https://nextjs.org/docs/pages/building-your-application/upgrading
Serve with turbopack Always Adapt package.json https://nextjs.org/docs/architecture/turbopack Might not work for all projects
Optimize Imports Importing from large Add setting in next config https://nextjs.org/docs/app/api-reference/next-config-js/optimizePackageImports Applies to some packages by default (only if setting is in file since its experimental?)
Modularize Imports Deprecated / https://nextjs.org/docs/architecture/nextjs-compiler#modularize-imports Use "Optimize Imports" instead, that does not need manual configuration
Disable Windows Defender Slow filesystem reads on Windows Computers Might not be possible on managed computers
Filesystem slowness overall is what we've seen as the cause of problems, e.g. with Docker ? ?

Maybe there is a chance you add something to the next documentation about this?

And then back to my problem:

  • We tried all optimizations we could find
  • We tried the optimizations you suggested
  • We cannot use turopack because of a bug with server relative imports
  • We are still having a very bad dx. Is there anything we can do to move forward from here?

@chriscaruth
Copy link

chriscaruth commented Aug 9, 2024

@timneutkens Here's our performance gist:

https://gist.github.com/chriscaruth/8e427460580cf44db32a17fad85f31bd (updated)

We are using next@14.2.4 due to next@canary requiring react@19.

  • My machine is M3 MBP/36GB Ram.
  • Seeing route compile times from ~3s - 8s

@PILLOWPET
Copy link

@PILLOWPET thanks for sharing the trace, we'll take a look! Note that it's currently expected that if you have a certain amount of code the memory usage is higher than webpack, this is something we're working on optimizing further. See https://nextjs.org/blog/next-14-2#turbopack-for-development-release-candidate

Hi @timneutkens, have you had the time to take a look at our trace ?

@DenisBessa
Copy link

@timneutkens Thanks for taking a look! I think you're right; I think the slowness comes from the Vanilla Extract plugin, probably in combination with Mantine that we're using. I've seen quite a few issues regarding this in their repository. I tried adding optimizePackageImports: ['@mantine/core', '@mantine/dropzone', '@mantine/tiptap'], but it didn't seem to help a lot. But at this point, it seems more like a webpack issue to debug rather than a NextJS one, so I suppose I'll be taking a look into that next.

Thanks for taking the time to look into this. I really appreciate it! By the way, is there an instruction somewhere on how to load those traces into Jaeger and debug by ourselves?

I'm having trouble with Mantine as well. Compilation takes 60-70 seconds for a single page. Were you able to discover something?

@wadamek65
Copy link

@DenisBessa Unfortunately, nothing ground-breaking, no. I did see a lot of issues regarding performance on both vanilla-extract and Mantine repositories, but none of the proposed solutions changed anything for us. I wanted to try to profile our webpack build via the ProfilingPlugin, but every time I did, the result was corrupted, so I wasn't able to find the root problem. I also tried out BundleAnalyzer but that also didn't give me a lot of helpful info.

I think, in this case, it would be more worthwhile debugging the webpack build itself rather than NextJS configuration because I think that's where the problem lies. If I discover anything, I'll let you know.

ForsakenHarmony pushed a commit that referenced this issue Aug 14, 2024
Implements support for running the Turbopack trace server, which is the
websocket server that powers https://turbo-trace-viewer.vercel.app/ when
using `NEXT_TURBOPACK_TRACING=1 NEXT_TURBOPACK_TRACE_SERVER=1`.

Currently you have to manually run the server through the Turbo
repository which in practice means that only people working on Turbopack
are able to run it.

With the bindings implemented anyone should be able to run the trace
server.

Note that the traces that come out of Turbopack are very low level,
they're meant for optimizing Turbopack like finding slowdowns / large
memory usage / optimizing performance.

However, it's useful for people that want to peek into why their
application is slow to compile. I.e. we've used
https://turbo-trace-viewer.vercel.app to investigate reports in #48748.

This PR adds support for `trace.log` by default, so if you add
`NEXT_TURBOPACK_TRACING=1 NEXT_TURBOPACK_TRACE_SERVER=1` it will
automatically select the `trace.log` for the current instance of
Next.js. You can only have one trace server running at the same time.

In order to support running the trace server standalone, which is useful
for investigating trace files other people have shared, I've added a new
subcommand `internal` that is not covered by semver / use at your own
risk. It's meant for internal tools that are useful to be bound to the
version of Next.js, the turbo-trace-server is a great example of that as
it has an internal binary format for storing data that needs to match
the trace.log file.

If you want to take a look at `.next/trace` instead the new `next
internal` subcommand can be used for that:

```sh
next internal turbo-trace-server [path]
```

For example:

```sh
next internal turbo-trace-server ~/Downloads/trace
```

Currently the trace server does not support loading multiple files, just
hasn't been implemented yet. Once we can load two or more files we can
load both `.next/trace` and `trace.log` when
`NEXT_TURBOPACK_TRACE_SERVER=1` and support multiple paths passed to
`next internal turbo-trace-server`.

PR includes a Turbopack upgrade:

* vercel/turborepo#8073 <!-- OJ Kwon -
feat(webpack-loaders): support dummy span interface -->
* vercel/turborepo#8083 <!-- OJ Kwon - fix(webpack):
print resource, project path when relative calc fails -->
* vercel/turborepo#8094 <!-- Tim Neutkens -
Implement bindings for Turbopack trace server -->

<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:

- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide

- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md

- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md

- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md

- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change

Closes NEXT-
Fixes #

-->

Closes NEXT-3328
@DenisBessa
Copy link

@DenisBessa Unfortunately, nothing ground-breaking, no. I did see a lot of issues regarding performance on both vanilla-extract and Mantine repositories, but none of the proposed solutions changed anything for us. I wanted to try to profile our webpack build via the ProfilingPlugin, but every time I did, the result was corrupted, so I wasn't able to find the root problem. I also tried out BundleAnalyzer but that also didn't give me a lot of helpful info.

I think, in this case, it would be more worthwhile debugging the webpack build itself rather than NextJS configuration because I think that's where the problem lies. If I discover anything, I'll let you know.

I've spend a few hours on this, and was able to reduce the compilation time, in my case, from 60 - 70 seconds to 18 seconds (without using TurboPack).

  1. Added mantine-react-table to optimizePackageImports;
  2. Added the following packages to serverExternalPackages: 'pdfjs-dist', 'xlsx', '@azure', '@opentelemetry', '@sentry', 'node-forge', 'pdf-lib', 'jose'.

I'm not sure abou the impacts from these changes in production, so for now they are enabled only in my dev environment.

I hope this can be useful.

ForsakenHarmony pushed a commit that referenced this issue Aug 15, 2024
Implements support for running the Turbopack trace server, which is the
websocket server that powers https://turbo-trace-viewer.vercel.app/ when
using `NEXT_TURBOPACK_TRACING=1 NEXT_TURBOPACK_TRACE_SERVER=1`.

Currently you have to manually run the server through the Turbo
repository which in practice means that only people working on Turbopack
are able to run it.

With the bindings implemented anyone should be able to run the trace
server.

Note that the traces that come out of Turbopack are very low level,
they're meant for optimizing Turbopack like finding slowdowns / large
memory usage / optimizing performance.

However, it's useful for people that want to peek into why their
application is slow to compile. I.e. we've used
https://turbo-trace-viewer.vercel.app to investigate reports in #48748.

This PR adds support for `trace.log` by default, so if you add
`NEXT_TURBOPACK_TRACING=1 NEXT_TURBOPACK_TRACE_SERVER=1` it will
automatically select the `trace.log` for the current instance of
Next.js. You can only have one trace server running at the same time.

In order to support running the trace server standalone, which is useful
for investigating trace files other people have shared, I've added a new
subcommand `internal` that is not covered by semver / use at your own
risk. It's meant for internal tools that are useful to be bound to the
version of Next.js, the turbo-trace-server is a great example of that as
it has an internal binary format for storing data that needs to match
the trace.log file.

If you want to take a look at `.next/trace` instead the new `next
internal` subcommand can be used for that:

```sh
next internal turbo-trace-server [path]
```

For example:

```sh
next internal turbo-trace-server ~/Downloads/trace
```

Currently the trace server does not support loading multiple files, just
hasn't been implemented yet. Once we can load two or more files we can
load both `.next/trace` and `trace.log` when
`NEXT_TURBOPACK_TRACE_SERVER=1` and support multiple paths passed to
`next internal turbo-trace-server`.

PR includes a Turbopack upgrade:

* vercel/turborepo#8073 <!-- OJ Kwon -
feat(webpack-loaders): support dummy span interface -->
* vercel/turborepo#8083 <!-- OJ Kwon - fix(webpack):
print resource, project path when relative calc fails -->
* vercel/turborepo#8094 <!-- Tim Neutkens -
Implement bindings for Turbopack trace server -->

<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:

- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide

- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md

- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md

- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md

- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change

Closes NEXT-
Fixes #

-->

Closes NEXT-3328
ForsakenHarmony pushed a commit that referenced this issue Aug 16, 2024
Implements support for running the Turbopack trace server, which is the
websocket server that powers https://turbo-trace-viewer.vercel.app/ when
using `NEXT_TURBOPACK_TRACING=1 NEXT_TURBOPACK_TRACE_SERVER=1`.

Currently you have to manually run the server through the Turbo
repository which in practice means that only people working on Turbopack
are able to run it.

With the bindings implemented anyone should be able to run the trace
server.

Note that the traces that come out of Turbopack are very low level,
they're meant for optimizing Turbopack like finding slowdowns / large
memory usage / optimizing performance.

However, it's useful for people that want to peek into why their
application is slow to compile. I.e. we've used
https://turbo-trace-viewer.vercel.app to investigate reports in #48748.

This PR adds support for `trace.log` by default, so if you add
`NEXT_TURBOPACK_TRACING=1 NEXT_TURBOPACK_TRACE_SERVER=1` it will
automatically select the `trace.log` for the current instance of
Next.js. You can only have one trace server running at the same time.

In order to support running the trace server standalone, which is useful
for investigating trace files other people have shared, I've added a new
subcommand `internal` that is not covered by semver / use at your own
risk. It's meant for internal tools that are useful to be bound to the
version of Next.js, the turbo-trace-server is a great example of that as
it has an internal binary format for storing data that needs to match
the trace.log file.

If you want to take a look at `.next/trace` instead the new `next
internal` subcommand can be used for that:

```sh
next internal turbo-trace-server [path]
```

For example:

```sh
next internal turbo-trace-server ~/Downloads/trace
```

Currently the trace server does not support loading multiple files, just
hasn't been implemented yet. Once we can load two or more files we can
load both `.next/trace` and `trace.log` when
`NEXT_TURBOPACK_TRACE_SERVER=1` and support multiple paths passed to
`next internal turbo-trace-server`.

PR includes a Turbopack upgrade:

* vercel/turborepo#8073 <!-- OJ Kwon -
feat(webpack-loaders): support dummy span interface -->
* vercel/turborepo#8083 <!-- OJ Kwon - fix(webpack):
print resource, project path when relative calc fails -->
* vercel/turborepo#8094 <!-- Tim Neutkens -
Implement bindings for Turbopack trace server -->

<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:

- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide

- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md

- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md

- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md

- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change

Closes NEXT-
Fixes #

-->

Closes NEXT-3328
@talglobus
Copy link

@timneutkens If this is still a standing offer, would greatly appreciate a look if possible
gist

@Kashish280598
Copy link

Hello 👋 @timneutkens ,

Here is the requested gist
📎 next.config , tailwind.config , trace

My development server is extremely slow taking 20 sec for content related change

@timneutkens
Copy link
Member

@Kashish280598 As you can see in the screenshot below there's a lot of "missing time" on these recompiles, that indicates webpack customization could be accounting for all that missing time as that customization is not instrumented. Looking at the next.config.js it seems the only customization is adding the svgr/loader... so maybe it's that?
CleanShot 2024-08-20 at 20 55 54@2x

@timneutkens
Copy link
Member

@talglobus This look suspicious 😄
CleanShot 2024-08-20 at 20 58 25@2x

Besides that seeing a lot of missing time spent, same as with @Kashish280598's trace, which indicates webpack customization, however there's no next.config.js included so can't verify that.

CleanShot 2024-08-20 at 20 59 37@2x

@talglobus
Copy link

@timneutkens

@talglobus This look suspicious 😄 CleanShot 2024-08-20 at 20 58 25@2x

Besides that seeing a lot of missing time spent, same as with @Kashish280598's trace, which indicates webpack customization, however there's no next.config.js included so can't verify that.

CleanShot 2024-08-20 at 20 59 37@2x

Here's our next.config.js, our tsconfig.json, and our tailwind.config.js just for completeness. Really appreciate the insight, and let me know if there's anything we can offer in thanks, you're saving a huge amount of developer productivity for us!

@timneutkens
Copy link
Member

@talglobus Looking at the config you're adding Sentry here: https://gist.github.com/talglobus/73cc83e1e136f1746af63b38cdb6c49a#file-next-config-js-L110 which adds a ton of overhead because Sentry's customization runs Rollup on all your code

@kl-thamm
Copy link

@talglobus Looking at the config you're adding Sentry here: https://gist.github.com/talglobus/73cc83e1e136f1746af63b38cdb6c49a#file-next-config-js-L110 which adds a ton of overhead because Sentry's customization runs Rollup on all your code

@timneutkens Does that mean we cannot use Sentry with Next? I am still trying to figure out what are just general performance improvements and what is a genuine next problem and how to fix that. This is another issue where its leaving it unclear to me. I have used Sentry in 4 different projects (one vue, python backend, react and now next) and yes - it does add some overhead to all these projects, but for all frontends I still have sub second compile times in dev and meanwhile up to 20 seconds with next. So this is not for scrubbing of the last few ms to get some more joy while developing, for many writing here, this is about saving hours in development time and getting a usable development environment.

Did you have a chance to look at this? #48748 (comment)
Is there any intention in addressing this on a higher level or have the issues been resolved from next's perspective and that is why you are resorting to individual advice? I am just a bit confused what the state and intention is with this issue.

@Kashish280598
Copy link

@Kashish280598 As you can see in the screenshot below there's a lot of "missing time" on these recompiles, that indicates webpack customization could be accounting for all that missing time as that customization is not instrumented. Looking at the next.config.js it seems the only customization is adding the svgr/loader... so maybe it's that? CleanShot 2024-08-20 at 20 55 54@2x

We had the following two customizations in our next.config.mjs for the Webpack configuration:

  1. @svgr/webpack
  2. file-loader for .mp3 files

Since we no longer use the 2nd file-loader customization, I removed it from the project as shown in this gist. However, I didn't notice any performance improvements after its removal. I’m wondering if @svgr/webpack might be causing the performance issues instead 🤔 ?

experimental: {
  turbo: {
    loaders: {
      ".svg": ["@svgr/webpack"],
    },
  },
},

Can you suggest some solution for this ? i found suggestion adding turbo like shown in the above. will that be helpful ?

@codedpills
Copy link

Hi @timneutkens, would appreciate if you could look into this trace if possible as it takes 20s+ to load the page on my dev server.

@yuvalkarmi
Copy link

yuvalkarmi commented Aug 23, 2024

Hey @timneutkens I actually ran trace-to-tree.mjs from the next.js project on my trace file and this is what I get. I don't know why my simple blog page is loading like 12K modules. I am using @mui/material but I do have:

  modularizeImports: {
    '@mui/icons-material': {
      transform: '@mui/icons-material/{{member}}',
    },
    '@mui/material': {
      transform: '@mui/material/{{member}}',
    },
    '@mui/lab': {
      transform: '@mui/lab/{{member}}',
    },
  },

Here's the output of trace-to-tree.mjs - would really appreciate understanding what's taking the longest here (and how you're doing that analysis).

Here's the tree.
blog-tree.txt

Would greatly appreciate your help in analyzing this!

@CarlosVergikosk

This comment was marked as spam.

@w01fgang

This comment was marked as resolved.

@gjolund

This comment was marked as off-topic.

@brendanmorrell
Copy link

brendanmorrell commented Aug 27, 2024

Hello, @timneutkens . Here is the trace (including next config and trace.log drive link for turbopack).

I use a typescript custom server/i don't use next dev, which i assume is why NEXT_CPU_PROF=1 did not create anr profiling files, or i would have included those.

I also just switched to turbopack, which helped a little, but the compile times are still ~20s per page. I attempted to put as may relevant packages in optimizePackageImports but if anything that appeared to make it worse, and since i could not figure out from trace-to-tree what the actual offender/s may be, i just removed it since it was slowing things down.

I also do use tailwindcss, but i tried removing it entirely from the project, and i did not see any improvement, so that is likely not the problem. i am including that/postcss files as well though in case it helps

Any help is appreciated. Thank you!

@grahamplace
Copy link

@timneutkens following up on Chris's ask here (we work together) — would be great to get your analysis of our trace if you have time 🙏 Thanks!

#48748 (comment)

@timneutkens
Copy link
Member

@brendanmorrell can you make sure to run Turbopack with next@canary, since you're on 14.2.x it's missing hundreds of fixes including performance improvements in Turbopack.

@grahamplace Similar for your app, it's on 14.2.4, but also the trace.log file is missing which means I can't inspect it.

@CarlosVergikosk
Copy link

CarlosVergikosk commented Aug 28, 2024

We cannot update to canary since it requires react 19 and some packages (mdx.js) do not support it.

@timneutkens
Copy link
Member

@CarlosVergikosk can you share a reproduction? We're running MDX in many places at Vercel and all our apps run on Next.js canary so maybe you're on an older version of MDX?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
linear: next Confirmed issue that is tracked by the Next.js team.
Projects
None yet
Development

No branches or pull requests