rebis-dev: Why 3402 commits ahead of, 3394 commits behind master?

[triska]

When I visit https://github.com/mthom/scryer-prolog/tree/rebis-dev, github shows:

This branch is 3402 commits ahead of, 3396 commits behind master.

Why such unexpectedly high numbers? The two branches share almost all commits, I expect rebis-dev to contain almost all commits from master, with possibly a few rare exceptions of recent commits that are no longer relevant for rebis-dev. Note that master itself has only 3884 commits, and rebis-dev has 3890 commits at the moment.

[mthom]

I created the abolish_2 branch nearly a year ago to begin scrubbing Term from the codebase, then rebased the pstr_in_heap branch on top of it.. at some point a discrepancy in the commit history was introduced.

[triska]

Still, it seems that the branches should/could have many more commits in common. Maybe this can be reflected somehow to make it clearer what the branches actually have in common. Maybe you @Skgland know how to do this? I would greatly appreciate if you could look into this.

[Skgland]

Oh no, this looks bad.

Based on a a visualized commit graph master and rebis-dev diverged after 80def45, which was committed 2018-09-28, and I can't see a merge between those branches either way since.

The main problem I see are all the things from master that appear to have been rebased onto rebis-dev that are complicating the history due to them necessarily getting a new commit id in the process.
A rebase of rebis-dev onto master or a merge with master wouldn't have had this problem.
I don't know a way to fix this automatically or easily.

The only thing I can come up with that might work, but which will definitely be tedious, would be to

1. checkout the most recent commit after the divergence that is just before the first rebased commit
2. merge the original commits from master rather than rebasing them
   • ensuring the merged result is identical to the rebased result
3. rebase the remaining rebis-dev history onto this newly created merge commit
4. repeat until done

[Skgland]

Not that it helps, but maybe abolish_2 was created from an already diverged branch rather than from master?

[bakaq]

Can't we just make a new branch from master and then cherrypick the commits that matter from the current rebis-dev? That would also involve a bunch of merge conflicts, but I think it would be way less painful than the whole rebase dance.

[triska]

Thank you for all suggestions and comments! Almost everything would be better than the situation we have now, where the branches are presented almost as if they don't have anything in common. We have many issues and discussions that link to commits in master, and it seems worthwhile to keep as many of them working as possible. Even if we can only preserve the existing commits in master up to somewhere in 2023, it would already be a huge improvement.

P.S.: Especially since in 2023, we already had a rebis-dev branch that closely followed master, maybe this could serve as the basis.

[triska]

The following may be a way out: We can simply add all rebis-dev changes in a single commit on top of master, by performing the following sequence of commands which I post here and explain in the following:

$ git fetch origin
$ git checkout master && git commit -a
$ git diff origin/master origin/rebis-dev | git apply
$ git add src/functor_macro.rs
$ git commit -a
$ git push origin master:rebis-dev --force
$ git reset --hard HEAD^

Explanation

First, make sure everything is up to date, master is checked out and all local changes are committed to the repository:

$ git fetch origin
$ git checkout master && git commit -a

Then, apply all changes of rebis-dev verbatim to master, i.e., do not attempt to apply the commits (which do not apply cleanly) but the textual changes to files:

$ git diff origin/master origin/rebis-dev | git apply

At this point, I get the following messages:

src/rebis.patch:19011: trailing whitespace.
                    (HeapCellValueTag::Cons | HeapCellValueTag::Fixnum | // HeapCellValueTag::Char | 
src/rebis.patch:22079: space before tab in indent.
                       if h == focus {
src/rebis.patch:22080: space before tab in indent.
                           break;
src/rebis.patch:22081: space before tab in indent.
                       } else {
src/rebis.patch:22082: space before tab in indent.
                           focus = h;
warning: squelched 13 whitespace errors
warning: 18 lines add whitespace errors.

I have ignored these messages, they do not influence the end result.

At this point, all changes of rebis-dev are applied to the repository, but not yet committed.

To these changes, add the file src/functor_macro.rs, which is new in rebis-dev and now also available in the repository:

$ git add src/functor_macro.rs

Now, commit all these changes with:

$ git commit -a

This yields a single big commit (69 files changed, ca. 10_000 insertions and deletions) on top of master.

The commit will look like this: triska@bd919dd

The result (i.e., master with this commit applied) is exactly identical to the remote rebis-dev branch.

Verification:

$ git diff origin/rebis-dev
$ (i.e., empty diff)

At last, push master to the remote rebis-dev branch:

$ git push origin master:rebis-dev --force

And then reset master to its original state by undoing the topmost commit which contains the rebis-dev changes:

$ git reset --hard HEAD^

I think the result is acceptable, much better than destroying the existing commit history of several years, especially since commit IDs are also used in commit messages and discussions that refer to them.

[Skgland]

The following may be a way out: We can simply add all rebis-dev changes in a single commit on top of master, by performing the following sequence of commands which I post here and explain in the following:

$ git fetch origin
$ git checkout master && git commit -a
$ git diff origin/master origin/rebis-dev | git apply
$ git add src/functor_macro.rs
$ git commit -a
$ git push origin master:rebis-dev --force
$ git reset --hard HEAD^

Explanation

First, make sure everything is up to date, master is checked out and all local changes are committed to the repository:

$ git fetch origin
$ git checkout master && git commit -a
Then, apply all changes of rebis-dev verbatim to master, i.e., do not attempt to apply the commits (which do not apply cleanly) but the textual changes to files:

$ git diff origin/master origin/rebis-dev | git apply
At this point, I get the following messages:

src/rebis.patch:19011: trailing whitespace.
                    (HeapCellValueTag::Cons | HeapCellValueTag::Fixnum | // HeapCellValueTag::Char | 
src/rebis.patch:22079: space before tab in indent.
                       if h == focus {
src/rebis.patch:22080: space before tab in indent.
                           break;
src/rebis.patch:22081: space before tab in indent.
                       } else {
src/rebis.patch:22082: space before tab in indent.
                           focus = h;
warning: squelched 13 whitespace errors
warning: 18 lines add whitespace errors.

I have ignored these messages, they do not influence the end result.

At this point, all changes of rebis-dev are applied to the repository, but not yet committed.

To these changes, add the file src/functor_macro.rs, which is new in rebis-dev and now also available in the repository:

$ git add src/functor_macro.rs

Now, commit all these changes with:

$ git commit -a
This yields a single big commit (69 files changed, ca. 10_000 insertions and deletions) on top of master.

The commit will look like this: triska@bd919dd

The result (i.e., master with this commit applied) is exactly identical to the remote rebis-dev branch.

Verification:

$ git diff origin/rebis-dev
$ (i.e., empty diff)
At last, push master to the remote rebis-dev branch:

$ git push origin master:rebis-dev --force

And then reset master to its original state by undoing the topmost commit which contains the rebis-dev changes:

$ git reset --hard HEAD^

I think the result is acceptable, much better than destroying the existing commit history of several years, especially since commit IDs are also used in commit messages and discussions that refer to them.

This generates the diff between master and rebis-dev,
then applies it to master,
this obviously ends in the state of rebis-dev.
But it also completely ignores/forcefully undoes all changes in master that are not in rebis-dev.

[triska]

But it also completely ignores/forcefully undoes all changes in master that are not in rebis-dev.

To clarify: After these steps, master is unchanged! The only purpose of these changes is to more closely align the history of rebis-dev with that of the existing master branch. At the end of these steps, only the history of rebis-dev is changed, and the content of rebis-dev is exactly the same as now!

At the end, we are conceptually in exactly the same position as we are now: There are new changes in rebis-dev (now all in a single commit), and master is the same as now.

Any changes that happened in the last few weeks must be integrated when (eventually) merging rebis-dev, exactly the same as now, such as a change to the README that took place a few days ago and is currently not yet in rebis-dev.