Hey Marcus, can you just add a License file to each of our repositories?
All of them?
Yeah, all 42…
This was a task given to me about 50 minutes ago. I’m done now.
Obviously I spent all that time writing a script to do this. And I wanted to share this with you guys and my future self.
Obviously I learned a lot as well.
Game plan
My game plan was pretty easy:
- Create a
LICENSE
with the proper text. - Clone all repositories of the organisation to a new folder
- Loop over all the repositories and add the
LICENSE
file to each of them - Add and commit to git and then push to the remote repository
Create a static license file
I went to Choose A License , picked an appropiate license and create a file out of that text. I ensured to update it with our company name.
Clone all repository
First part was pretty each to find and I just ripped this one-liner:
curl -u [[USERNAME]] -s https://api.github.com/orgs/[[ORGANIZATION]]/repos?per_page=200 | ruby -rubygems -e 'require "json"; JSON.load(STDIN.read).each { |repo| %x[git clone #{repo["ssh_url"]} ]}'
Let’s walk through this. First the simple things:
- I replaced
[[USERNAME]]
with my usernamemarcusoftnet
- I replaced
[[ORGANIZATION]]
with my org … that was secret. Let call itMarcusCo
for now.
This created a script that looks like this:
curl -u marcusoftnet -s https://api.github.com/orgs/marcusco/repos?per_page=200 | ruby -rubygems -e 'require "json"; JSON.load(STDIN.read).each { |repo| %x[git clone #{repo["ssh_url"]} ]}'
I then test ran that first part curl -u marcusoftnet -s https://api.github.com/orgs/marcusco/repos?per_page=200
that just barfed a lot of JSON information about all my repos to the terminal.
That was good because that meant that I understood the second part much easier:
- With Ruby were using a gem call
json
. - That lets us load that barfed json that
curl
command gave us. - We loop over that array of results, i.e. the repositories
- For each (
.each { ... }
) we simply do agit clone
and use the value in thessh_url
- That actuall didn’t really work for me so I changed it into
clone_url
- That actuall didn’t really work for me so I changed it into
All in all I ended up with this script:
curl -u marcusoftnet -s https://api.github.com/orgs/marcusco/repos?per_page=200 | ruby -rubygems -e 'require "json"; JSON.load(STDIN.read).each { |repo| %x[git clone #{repo["clone_url"]} ]}'
Running this would bring down all the repositories to the current directory.
Great - on to the next part
Loop over directories and add a License file to each
This part could most definitely been done more effective, in lines of codes, but I actually understand this part now and it works. Good enough:
for dir in ./* ; do
if [ -d "$dir" ]; then
dir=${dir%*/}
echo "Copying LICENCE"
cp ~/cloner/LICENSE ${dir##*/}
fi
done
- That
for dir in ./* ; do
loops over all the files in the current directory - The
if [ -d "$dir" ]; then
checks that this is a directory - I then pick out the directory name to a variable called
dir
- Finally I just copy the
LICENSE
file, at a well known location to this directory
Done. File in place.
Git add, commit and push
Now all I needed to do was to add this to git, then commit it and finally push it.
I did this in that same loop. Could probably have been done differently but again, I get this and it works.
Here are the commands I used:
echo "Doing git stuff"
cd ${dir##*/}
git add .
git commit -m "Added MIT license file"
git push
cd ..
- I first
cd
into the directory in question, where theLICENSE
just was copied to. git add .
will add the file to the staging areagit commit -m
commits the file with a nice messagegit push
push the file to the remote- And then I
cd
back out of the repository for the nex lap in the loop
House keeping
On the top of the script I create a new directory where all of these actions will take place. I’ve found this very useful to be able to delete everything to start fresh.
mkdir tmpClone
cd tmpClone
At the end of the script I then list all the LICENSE
in the sub-directories just to see what got created.
cd ..
ls tmpClone/**/LICENSE
Full script
The full script looks like this:
#!/bin/bash
mkdir tmpClone
cd tmpClone
echo "Clone repos"
curl -u marcusoftnet -s https://api.github.com/orgs/marcusco/repos?per_page=200 | ruby -rubygems -e 'require "json"; JSON.load(STDIN.read).each { |repo| %x[git clone #{repo["clone_url"]} ]}'
echo "Repos cloned and ready"
for dir in ./* ; do
if [ -d "$dir" ]; then
dir=${dir%*/}
echo "Copying LICENCE"
cp ~/Downloads/cloner/LICENSE ${dir##*/}
echo "Doing git stuff"
cd ${dir##*/}
git add .
git commit -m "Added MIT license file"
git push
cd ..
fi
done
echo "Here's the files I've added"
cd ..
ls tmpClone/**/LICENSE
I learned a lot by doing this - the number one thing is that writing script (and a blog post about it) is often faster than to do it by hand…