Repeated "Downloading Sources and Javadoc"

[kohlschuetter]

Using latest Eclipse 2024-06 (4.32.0) on macOS, also happening with latest m2e-core from master

I keep seeing "Downloading Sources and Javadoc" appear in the "Progress" view for projects that aren't actually modified.
This would appear even when downloading sources/javadocs is disabled in settings.

I realized that there are several mentions of this behavior in older tickets (e.g., #252 and #123), so I checked the source code to see what's going on.

Apparently, the culprit for this behavior currently seems to be the following code:

m2e-core/org.eclipse.m2e.jdt/src/org/eclipse/m2e/jdt/internal/BuildPathManager.java

Lines 300 to 313 in 7755962

	ArtifactKey aKey = desc.getArtifactKey();
	if(aKey != null) { // maybe we should try to find artifactKey little harder here?
	boolean isSnapshot = aKey.version().endsWith("-SNAPSHOT");
	// We should update a sources/javadoc jar for a snapshot in case they're already downloaded.
	File mainFile = desc.getPath() != null ? desc.getPath().toFile() : null;
	File srcFile = srcPath != null ? srcPath.toFile() : null;
	boolean downloadSources = (srcPath == null && mavenConfiguration.isDownloadSources())
	|| (isSnapshot && isLastModifiedBefore(srcFile, mainFile));
	File javaDocFile = javaDocUrl != null ? getAttachedArtifactFile(aKey, CLASSIFIER_JAVADOC) : null;
	boolean downloadJavaDoc = (javaDocUrl == null && mavenConfiguration.isDownloadJavaDoc())
	|| (isSnapshot && isLastModifiedBefore(javaDocFile, mainFile));
	scheduleDownload(facade.getProject(), facade.getMavenProject(monitor), aKey, downloadSources,
	downloadJavaDoc);
	}

This checks if sources/javadocs should be downloaded. There is some special-case that forces downloading sources/javadocs if the "main" artifact (jar, etc.) in the local m2 cache is

1. a SNAPSHOT version, and
2. newer than the corresponding source / javadoc jars. "Newer" is checked using File.lastModified.

It now so happens for some artifacts that the source/javadoc.jars are a few milliseconds older than the main artifact (although in some case they were a few seconds apart). This may be the reason why it's not always reproducible — on filesystems with a timestamp granularity of seconds (e.g., HFS+, ext3) instead of milliseconds (e.g., APFS, zfs), the comparison will tell us that the timestamp is identical...

One may think that this gets resolved automatically. However, when there's no newer SNAPSHOT build to be downloaded from any configured repository (usually because this artifact only exists locally, or simply no newer builds were available), then this results in an ever-going attempt of trying to download these files...

To reproduce, the following should trigger the bug:

1. Create a new Maven project with a 1.0-SNAPSHOT version.
2. "mvn clean install" the version to the local m2 repository cache
3. Import the project to Eclipse
4. manually "touch" the corresponding main .jar artifact in ~/.m2/repository/.../artifact-1.0-SNAPSHOT.jar
5. Create another m2e artifact that depends on the former artifact
6. Watch the "Progress" view in Eclipse

To fix this, we will have to somehow keep track of when we last checked for these snapshot versions.

What worked for me (as a proof of concept) was to check the last-modified timestamp of "m2e-lastUpdated.properties", and only perform the update the main .jar artifact was newer than that. I also added check around scheduleDownload so it's only called if either downloadSources or downloadJavaDoc are true, or when the mainFile is missing.

        ArtifactKey aKey = desc.getArtifactKey();
        if(aKey != null) { // maybe we should try to find artifactKey little harder here?
          boolean isSnapshot = aKey.version().endsWith("-SNAPSHOT");
          // We should update a sources/javadoc jar for a snapshot in case they're already downloaded.
          File mainFile = desc.getPath() != null ? desc.getPath().toFile() : null;
          File srcFile = srcPath != null ? srcPath.toFile() : null;
          File m2eLastUpdatedFile = (mainFile == null ? null : new File(mainFile.getParentFile(), "m2e-lastUpdated.properties"));
          boolean mayForceUpdate = isSnapshot && isLastModifiedBefore(m2eLastUpdatedFile, mainFile);
          boolean downloadSources = (srcPath == null && mavenConfiguration.isDownloadSources())
              || (mayForceUpdate && isLastModifiedBefore(srcFile, mainFile));
          File javaDocFile = javaDocUrl != null ? getAttachedArtifactFile(aKey, CLASSIFIER_JAVADOC) : null;
          boolean downloadJavaDoc = (javaDocUrl == null && mavenConfiguration.isDownloadJavaDoc())
              || (mayForceUpdate && isLastModifiedBefore(javaDocFile, mainFile));

           if (downloadSources || downloadJavaDoc || mainFile == null) {
            scheduleDownload(facade.getProject(), facade.getMavenProject(monitor), aKey, downloadSources,
              downloadJavaDoc);
          }
        }