[Cake] Cake tree unreadable

[Cake] Cake tree unreadable

[Kevin Darbyshire-Bryant]

In the nicest possible way… what on earth is going on with the cake source tree of late?  Many ‘cascaded’ merges make things impossible to read and stuff appears to be getting lost as well. e.g. the recent change "Switch ingress failsafe to 2/3 instead of 1/4 bandwidth.” has now appeared twice, the first time having somehow got lost.  What on earth is going on?

It’s as if there are at least 3 repos that have all got out of step with each other and now each time a merge is required from one it pulls in all the merges from the others.  May I suggest a ‘git fetch —all’, take a safety copy branch pointer of where you are now ‘git checkout -b safety’, switch back to where you were ‘git checkout cobalt’, reset the pointer and current work tree to upstream’s sane state ‘git reset —hard upstream/cobalt’ and finally a ‘git diff safety’ to see what’s missing between the two.  ‘git cherry-pick missing-commit’ to grab any missing commit/s determined by looking at the diff.

Re: [Cake] Cake tree unreadable

[Dave Taht]

Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> writes:

> In the nicest possible way… what on earth is going on with the cake source tree
> of late?  Many ‘cascaded’ merges make things impossible to read and stuff
> appears to be getting lost as well. e.g. the recent change "Switch ingress
> failsafe to 2/3 instead of 1/4 bandwidth.” has now appeared twice, the first
> time having somehow got lost.  What on earth is going on?
>
> It’s as if there are at least 3 repos that have all got out of step with each
> other and now each time a merge is required from one it pulls in all the merges
> from the others.  May I suggest a ‘git fetch —all’, take a safety copy branch
> pointer of where you are now ‘git checkout -b safety’, switch back to where you
> were ‘git checkout cobalt’, reset the pointer and current work tree to
> upstream’s sane state ‘git reset —hard upstream/cobalt’ and finally a ‘git diff
> safety’ to see what’s missing between the two.  ‘git cherry-pick missing-commit’
> to grab any missing commit/s determined by looking at the diff.

It sounds like your git-foo is stronger than ours! I'm not even trying
to get head to work, tho my intent would be to promote cobalt to it.

I didn't know about the cherry-pick option til now, either. I was doing
a git format-patch thenewcommit, then a git am on my other branch.

For example, with this config, git fetch --all doesn't do anything.

[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[remote "origin"]
        url = git@github.com:dtaht/sch_cake.git
        fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
        remote = origin
        merge = refs/heads/master
[branch "for_upstream_4.16"]
        remote = origin
        merge = refs/heads/for_upstream_4.16
[branch "cobalt"]
        remote = origin
        merge = refs/heads/cobalt

Re: [Cake] Cake tree unreadable

[Kevin Darbyshire-Bryant]

> On 28 Nov 2017, at 18:48, Dave Taht <dave@taht.net> wrote:
> 
> 
> 
> It sounds like your git-foo is stronger than ours! I'm not even trying
> to get head to work, tho my intent would be to promote cobalt to it.

git checkout master
git pull   (does the equivalent of git fetch origin; git merge origin/master)
git merge cobalt   (this will produce a minor merge conflict in sch_cake.c)

fix merge conflict

git add sch_cake.c
git commit   (complete the merge - and create a merge commit in process)

git diff cobalt - should return nothing…content of master & cobalt are the same.

git push  (send this to github assuming above is true!)

If it were me I’d now restart the cobalt ‘feature’ branch from this new ‘master’ point.

git checkout cobalt
git reset —hard master  (resets ‘cobalts’ commit pointer and the current tree on disc to where master is)
git push -f   (send that to github -f means force)

You’ve just created a ‘new’ history for the ‘cobalt’ feature branch, so all ‘clients’ of that branch will see a ‘forced update’ message….you’ve broken the linear git history for that branch, so they’d have to do something like ‘git checkout cobalt; git reset —hard origin/cobalt’

I used to be terrified of ‘git reset’ until I read https://git-scm.com/blog/2011/07/11/reset.html - at that point I realised all it (and  the whole of git) is about moving pointers contained in branch names.

Of course I’d advise checking things carefully before doing force pushes….but then I’d imagine many on this list have forks and hence clones of the git repo in various places so it’s all recoverable.


> 
> I didn't know about the cherry-pick option til now, either. I was doing
> a git format-patch thenewcommit, then a git am on my other branch.
I’ve just sped up your workflow then :-)
> 
> For example, with this config, git fetch --all doesn't do anything.
git fetch —all  goes and gets all the branches/commits etc from all configured remotes.  You’ve only 1 remote (origin…which points at github) and it’s likely that since it’s not very active that you’ve all the commits etc already in your local git database.

I have several remotes e.g.

[remote "origin"]
        url = git@github.com:ldir-EDB0/sch_cake.git
        fetch = +refs/heads/*:refs/remotes/origin/*
[remote "upstream"]
        url = git@github.com:dtaht/sch_cake.git
        fetch = +refs/heads/*:refs/remotes/upstream/*
[remote "teg"]
        url = git@github.com:tegularius/sch_cake.git
        fetch = +refs/heads/*:refs/remotes/teg/*
[remote "rmounce"]
        url = git@github.com:rmounce/sch_cake.git
        fetch = +refs/heads/*:refs/remotes/rmounce/*

so my ‘git fetch —all’ will go and look in all those places for things I’m missing.  Origin is my own github base cake repo (a clone/fork of yours), ‘upstream’ is a pointer directly to your github repo, ‘teg’ & ‘rmounce’ are pointers to their forks/clones.

So to bring my master up to date with yours:

git checkout master
git fetch —all
git merge upstream/master. (which should do a ‘fast-forward’ to where you are since I intentionally don’t do any of my own work in master)
git push (update my own fork with all the latest stuff)


My own stuff (not that there is any anymore ‘cos it’s all in cobalt) would then be rebased on top of the stable (but moving) master e.g:

git checkout worldpeace (my WIP mega solution branch)
edit edit edit, commit commit commit
git rebase master. - replay all of my stuff on top of the updated master
edit - fixup any conflicts
git push -f  (force update my worldpeace branch on github)


I like git, but I’m by no means a guru on it….and it took me about 2 years to go from ‘I hate it’ to ‘ahhh I get it - sort of’.  The git reset article helped.  Also having ‘git prompt’ enabled was a godsend.

> 
> [core]
>        repositoryformatversion = 0
>        filemode = true
>        bare = false
>        logallrefupdates = true
> [remote "origin"]
>        url = git@github.com:dtaht/sch_cake.git
>        fetch = +refs/heads/*:refs/remotes/origin/*
> [branch "master"]
>        remote = origin
>        merge = refs/heads/master
> [branch "for_upstream_4.16"]
>        remote = origin
>        merge = refs/heads/for_upstream_4.16
> [branch "cobalt"]
>        remote = origin
>        merge = refs/heads/cobalt
> 

Re: [Cake] Cake tree unreadable

[Dave Taht]

A flag day here is feasible. I will fiddle along the lines you describe.

As for other flag days...

I'm toying with the idea of fixing xstats in a separate branch. I really
hate the idea of breaking backward compatability here, but I do suspect
it will be a barrier to upstreaming, and it is, quite inefficient.


Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> writes:

>> On 28 Nov 2017, at 18:48, Dave Taht <dave@taht.net> wrote:
>> 
>> 
>> 
>> It sounds like your git-foo is stronger than ours! I'm not even trying
>> to get head to work, tho my intent would be to promote cobalt to it.
>
> git checkout master
> git pull   (does the equivalent of git fetch origin; git merge origin/master)
> git merge cobalt   (this will produce a minor merge conflict in sch_cake.c)
>
> fix merge conflict
>
> git add sch_cake.c
> git commit   (complete the merge - and create a merge commit in process)
>
> git diff cobalt - should return nothing…content of master & cobalt are the same.
>
> git push  (send this to github assuming above is true!)
>
> If it were me I’d now restart the cobalt ‘feature’ branch from this new ‘master’ point.
>
> git checkout cobalt
> git reset —hard master  (resets ‘cobalts’ commit pointer and the current tree on disc to where master is)
> git push -f   (send that to github -f means force)
>
> You’ve just created a ‘new’ history for the ‘cobalt’ feature branch, so all
> ‘clients’ of that branch will see a ‘forced update’ message….you’ve broken the
> linear git history for that branch, so they’d have to do something like ‘git
> checkout cobalt; git reset —hard origin/cobalt’
>
> I used to be terrified of ‘git reset’ until I read https://git-scm.com/blog/2011/07/11/reset.html - at that point I realised all it (and  the whole of git) is about moving pointers contained in branch names.
>
> Of course I’d advise checking things carefully before doing force pushes….but then I’d imagine many on this list have forks and hence clones of the git repo in various places so it’s all recoverable.
>
>
>> 
>> I didn't know about the cherry-pick option til now, either. I was doing
>> a git format-patch thenewcommit, then a git am on my other branch.
> I’ve just sped up your workflow then :-)
>> 
>> For example, with this config, git fetch --all doesn't do anything.
> git fetch —all goes and gets all the branches/commits etc from all configured
> remotes.  You’ve only 1 remote (origin…which points at github) and it’s likely
> that since it’s not very active that you’ve all the commits etc already in your
> local git database.
>
> I have several remotes e.g.
>
> [remote "origin"]
>         url = git@github.com:ldir-EDB0/sch_cake.git
>         fetch = +refs/heads/*:refs/remotes/origin/*
> [remote "upstream"]
>         url = git@github.com:dtaht/sch_cake.git
>         fetch = +refs/heads/*:refs/remotes/upstream/*
> [remote "teg"]
>         url = git@github.com:tegularius/sch_cake.git
>         fetch = +refs/heads/*:refs/remotes/teg/*
> [remote "rmounce"]
>         url = git@github.com:rmounce/sch_cake.git
>         fetch = +refs/heads/*:refs/remotes/rmounce/*
>
> so my ‘git fetch —all’ will go and look in all those places for things I’m
> missing.  Origin is my own github base cake repo (a clone/fork of yours),
> ‘upstream’ is a pointer directly to your github repo, ‘teg’ & ‘rmounce’ are
> pointers to their forks/clones.
>
> So to bring my master up to date with yours:
>
> git checkout master
> git fetch —all
> git merge upstream/master. (which should do a ‘fast-forward’ to where you are since I intentionally don’t do any of my own work in master)
> git push (update my own fork with all the latest stuff)
>
>
> My own stuff (not that there is any anymore ‘cos it’s all in cobalt) would then be rebased on top of the stable (but moving) master e.g:
>
> git checkout worldpeace (my WIP mega solution branch)
> edit edit edit, commit commit commit
> git rebase master. - replay all of my stuff on top of the updated master
> edit - fixup any conflicts
> git push -f  (force update my worldpeace branch on github)
>
>
> I like git, but I’m by no means a guru on it….and it took me about 2 years to go from ‘I hate it’ to ‘ahhh I get it - sort of’.  The git reset article helped.  Also having ‘git prompt’ enabled was a godsend.
>
>> 
>> [core]
>>        repositoryformatversion = 0
>>        filemode = true
>>        bare = false
>>        logallrefupdates = true
>> [remote "origin"]
>>        url = git@github.com:dtaht/sch_cake.git
>>        fetch = +refs/heads/*:refs/remotes/origin/*
>> [branch "master"]
>>        remote = origin
>>        merge = refs/heads/master
>> [branch "for_upstream_4.16"]
>>        remote = origin
>>        merge = refs/heads/for_upstream_4.16
>> [branch "cobalt"]
>>        remote = origin
>>        merge = refs/heads/cobalt
>> 

Re: [Cake] Cake tree unreadable

[Toke Høiland-Jørgensen]

Dave Taht <dave@taht.net> writes:

> A flag day here is feasible. I will fiddle along the lines you
> describe.

FWIW I don't think the history is that bad. Sure, there are a bunch of
merge commits, but picking out the real ones is not that difficult
(unless you are using the github web interface, I guess, but that sucks
for most things anyway). I'm not sure rewriting history in the existing
branch is worth the hassle. We're going to reset history anyway once
it's been upstreamed.

If you *are* going to rewrite history, please at least do it in a
separate branch instead of force pushing to an existing one...

> I'm toying with the idea of fixing xstats in a separate branch. I
> really hate the idea of breaking backward compatability here, but I do
> suspect it will be a barrier to upstreaming, and it is, quite
> inefficient.

I'm not sure this will actually be enough of a backwards compatibility
break to warrant the term 'flag day'. I.e., it won't affect outside
behaviour. Sure, you need to update userspace and the kernel module
together, but you kinda need to do that anyway when new features are
added. And for LEDE, we'll just push updates to both packages at the
same time, which should give new images and even `opkg update` a
consistent view...

So I'd say go ahead and break things. The worst thing that can happen is
that some users will lose the statistics until they update their
userspace tool...

-Toke

Re: [Cake] Cake tree unreadable

[Sebastian Moeller]

Hi All,


> On Nov 28, 2017, at 23:37, Dave Taht <dave@taht.net> wrote:
> 
> 
> A flag day here is feasible. I will fiddle along the lines you describe.
> 
> As for other flag days...
> 
> I'm toying with the idea of fixing xstats in a separate branch. I really
> hate the idea of breaking backward compatability here, but I do suspect
> it will be a barrier to upstreaming, and it is, quite inefficient.


	So if we go and change the statistics could we rename max_len to maxpacket so that fq_codel and cake report the same information under the same name. Sure the qdisc statistics are not terribly well coordinated naming-wise, but maybe cake could show to be a good citizen here?

Best Regards
> 
> 
> Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> writes:
> 
>>> On 28 Nov 2017, at 18:48, Dave Taht <dave@taht.net> wrote:
>>> 
>>> 
>>> 
>>> It sounds like your git-foo is stronger than ours! I'm not even trying
>>> to get head to work, tho my intent would be to promote cobalt to it.
>> 
>> git checkout master
>> git pull   (does the equivalent of git fetch origin; git merge origin/master)
>> git merge cobalt   (this will produce a minor merge conflict in sch_cake.c)
>> 
>> fix merge conflict
>> 
>> git add sch_cake.c
>> git commit   (complete the merge - and create a merge commit in process)
>> 
>> git diff cobalt - should return nothing…content of master & cobalt are the same.
>> 
>> git push  (send this to github assuming above is true!)
>> 
>> If it were me I’d now restart the cobalt ‘feature’ branch from this new ‘master’ point.
>> 
>> git checkout cobalt
>> git reset —hard master  (resets ‘cobalts’ commit pointer and the current tree on disc to where master is)
>> git push -f   (send that to github -f means force)
>> 
>> You’ve just created a ‘new’ history for the ‘cobalt’ feature branch, so all
>> ‘clients’ of that branch will see a ‘forced update’ message….you’ve broken the
>> linear git history for that branch, so they’d have to do something like ‘git
>> checkout cobalt; git reset —hard origin/cobalt’
>> 
>> I used to be terrified of ‘git reset’ until I read https://git-scm.com/blog/2011/07/11/reset.html - at that point I realised all it (and  the whole of git) is about moving pointers contained in branch names.
>> 
>> Of course I’d advise checking things carefully before doing force pushes….but then I’d imagine many on this list have forks and hence clones of the git repo in various places so it’s all recoverable.
>> 
>> 
>>> 
>>> I didn't know about the cherry-pick option til now, either. I was doing
>>> a git format-patch thenewcommit, then a git am on my other branch.
>> I’ve just sped up your workflow then :-)
>>> 
>>> For example, with this config, git fetch --all doesn't do anything.
>> git fetch —all goes and gets all the branches/commits etc from all configured
>> remotes.  You’ve only 1 remote (origin…which points at github) and it’s likely
>> that since it’s not very active that you’ve all the commits etc already in your
>> local git database.
>> 
>> I have several remotes e.g.
>> 
>> [remote "origin"]
>>        url = git@github.com:ldir-EDB0/sch_cake.git
>>        fetch = +refs/heads/*:refs/remotes/origin/*
>> [remote "upstream"]
>>        url = git@github.com:dtaht/sch_cake.git
>>        fetch = +refs/heads/*:refs/remotes/upstream/*
>> [remote "teg"]
>>        url = git@github.com:tegularius/sch_cake.git
>>        fetch = +refs/heads/*:refs/remotes/teg/*
>> [remote "rmounce"]
>>        url = git@github.com:rmounce/sch_cake.git
>>        fetch = +refs/heads/*:refs/remotes/rmounce/*
>> 
>> so my ‘git fetch —all’ will go and look in all those places for things I’m
>> missing.  Origin is my own github base cake repo (a clone/fork of yours),
>> ‘upstream’ is a pointer directly to your github repo, ‘teg’ & ‘rmounce’ are
>> pointers to their forks/clones.
>> 
>> So to bring my master up to date with yours:
>> 
>> git checkout master
>> git fetch —all
>> git merge upstream/master. (which should do a ‘fast-forward’ to where you are since I intentionally don’t do any of my own work in master)
>> git push (update my own fork with all the latest stuff)
>> 
>> 
>> My own stuff (not that there is any anymore ‘cos it’s all in cobalt) would then be rebased on top of the stable (but moving) master e.g:
>> 
>> git checkout worldpeace (my WIP mega solution branch)
>> edit edit edit, commit commit commit
>> git rebase master. - replay all of my stuff on top of the updated master
>> edit - fixup any conflicts
>> git push -f  (force update my worldpeace branch on github)
>> 
>> 
>> I like git, but I’m by no means a guru on it….and it took me about 2 years to go from ‘I hate it’ to ‘ahhh I get it - sort of’.  The git reset article helped.  Also having ‘git prompt’ enabled was a godsend.
>> 
>>> 
>>> [core]
>>>       repositoryformatversion = 0
>>>       filemode = true
>>>       bare = false
>>>       logallrefupdates = true
>>> [remote "origin"]
>>>       url = git@github.com:dtaht/sch_cake.git
>>>       fetch = +refs/heads/*:refs/remotes/origin/*
>>> [branch "master"]
>>>       remote = origin
>>>       merge = refs/heads/master
>>> [branch "for_upstream_4.16"]
>>>       remote = origin
>>>       merge = refs/heads/for_upstream_4.16
>>> [branch "cobalt"]
>>>       remote = origin
>>>       merge = refs/heads/cobalt
>>> 
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] Cake tree unreadable

[Dave Taht]

Sebastian Moeller <moeller0@gmx.de> writes:

> Hi All,
>
>
>> On Nov 28, 2017, at 23:37, Dave Taht <dave@taht.net> wrote:
>> 
>> 
>> A flag day here is feasible. I will fiddle along the lines you describe.
>> 
>> As for other flag days...
>> 
>> I'm toying with the idea of fixing xstats in a separate branch. I really
>> hate the idea of breaking backward compatability here, but I do suspect
>> it will be a barrier to upstreaming, and it is, quite inefficient.
>
>
> 	So if we go and change the statistics could we rename max_len to
> maxpacket so that fq_codel and cake report the same information under the same
> name. Sure the qdisc statistics are not terribly well coordinated naming-wise,
> but maybe cake could show to be a good citizen here?

The justification for that change was that it's 7 characters.

I really do want to open up the discussion of other stuff worth
breaking along the way.

>
> Best Regards
>> 
>> 
>> Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> writes:
>> 
>>>> On 28 Nov 2017, at 18:48, Dave Taht <dave@taht.net> wrote:
>>>> 
>>>> 
>>>> 
>>>> It sounds like your git-foo is stronger than ours! I'm not even trying
>>>> to get head to work, tho my intent would be to promote cobalt to it.
>>> 
>>> git checkout master
>>> git pull   (does the equivalent of git fetch origin; git merge origin/master)
>>> git merge cobalt   (this will produce a minor merge conflict in sch_cake.c)
>>> 
>>> fix merge conflict
>>> 
>>> git add sch_cake.c
>>> git commit   (complete the merge - and create a merge commit in process)
>>> 
>>> git diff cobalt - should return nothing…content of master & cobalt are the same.
>>> 
>>> git push  (send this to github assuming above is true!)
>>> 
>>> If it were me I’d now restart the cobalt ‘feature’ branch from this new ‘master’ point.
>>> 
>>> git checkout cobalt
>>> git reset —hard master  (resets ‘cobalts’ commit pointer and the current tree on disc to where master is)
>>> git push -f   (send that to github -f means force)
>>> 
>>> You’ve just created a ‘new’ history for the ‘cobalt’ feature branch, so all
>>> ‘clients’ of that branch will see a ‘forced update’ message….you’ve broken the
>>> linear git history for that branch, so they’d have to do something like ‘git
>>> checkout cobalt; git reset —hard origin/cobalt’
>>> 
>>> I used to be terrified of ‘git reset’ until I read https://git-scm.com/blog/2011/07/11/reset.html - at that point I realised all it (and  the whole of git) is about moving pointers contained in branch names.
>>> 
>>> Of course I’d advise checking things carefully before doing force pushes….but then I’d imagine many on this list have forks and hence clones of the git repo in various places so it’s all recoverable.
>>> 
>>> 
>>>> 
>>>> I didn't know about the cherry-pick option til now, either. I was doing
>>>> a git format-patch thenewcommit, then a git am on my other branch.
>>> I’ve just sped up your workflow then :-)
>>>> 
>>>> For example, with this config, git fetch --all doesn't do anything.
>>> git fetch —all goes and gets all the branches/commits etc from all configured
>>> remotes.  You’ve only 1 remote (origin…which points at github) and it’s likely
>>> that since it’s not very active that you’ve all the commits etc already in your
>>> local git database.
>>> 
>>> I have several remotes e.g.
>>> 
>>> [remote "origin"]
>>>        url = git@github.com:ldir-EDB0/sch_cake.git
>>>        fetch = +refs/heads/*:refs/remotes/origin/*
>>> [remote "upstream"]
>>>        url = git@github.com:dtaht/sch_cake.git
>>>        fetch = +refs/heads/*:refs/remotes/upstream/*
>>> [remote "teg"]
>>>        url = git@github.com:tegularius/sch_cake.git
>>>        fetch = +refs/heads/*:refs/remotes/teg/*
>>> [remote "rmounce"]
>>>        url = git@github.com:rmounce/sch_cake.git
>>>        fetch = +refs/heads/*:refs/remotes/rmounce/*
>>> 
>>> so my ‘git fetch —all’ will go and look in all those places for things I’m
>>> missing.  Origin is my own github base cake repo (a clone/fork of yours),
>>> ‘upstream’ is a pointer directly to your github repo, ‘teg’ & ‘rmounce’ are
>>> pointers to their forks/clones.
>>> 
>>> So to bring my master up to date with yours:
>>> 
>>> git checkout master
>>> git fetch —all
>>> git merge upstream/master. (which should do a ‘fast-forward’ to where you are since I intentionally don’t do any of my own work in master)
>>> git push (update my own fork with all the latest stuff)
>>> 
>>> 
>>> My own stuff (not that there is any anymore ‘cos it’s all in cobalt) would then be rebased on top of the stable (but moving) master e.g:
>>> 
>>> git checkout worldpeace (my WIP mega solution branch)
>>> edit edit edit, commit commit commit
>>> git rebase master. - replay all of my stuff on top of the updated master
>>> edit - fixup any conflicts
>>> git push -f  (force update my worldpeace branch on github)
>>> 
>>> 
>>> I like git, but I’m by no means a guru on it….and it took me about 2 years to go from ‘I hate it’ to ‘ahhh I get it - sort of’.  The git reset article helped.  Also having ‘git prompt’ enabled was a godsend.
>>> 
>>>> 
>>>> [core]
>>>>       repositoryformatversion = 0
>>>>       filemode = true
>>>>       bare = false
>>>>       logallrefupdates = true
>>>> [remote "origin"]
>>>>       url = git@github.com:dtaht/sch_cake.git
>>>>       fetch = +refs/heads/*:refs/remotes/origin/*
>>>> [branch "master"]
>>>>       remote = origin
>>>>       merge = refs/heads/master
>>>> [branch "for_upstream_4.16"]
>>>>       remote = origin
>>>>       merge = refs/heads/for_upstream_4.16
>>>> [branch "cobalt"]
>>>>       remote = origin
>>>>       merge = refs/heads/cobalt
>>>> 
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake