A few weeks ago the CentOS announce list and twitter feed introduced yet another repo the Continuous Release repo or cr for short. This new repo contains security and bug fixes from the upstream 5.7 release and I imagine will continue with further releases (5.8 & 5.9). CentOS has also promised a 6.0 Continuous Release repo, but have yet to deliver on that.
First off create the directories that will become the repositories.
mkdir -pv /Storage/yumRepo/{5,5.4,5.5/{os,updates,cr}/i386
mkdir -pv /Storage/yumRepo/local/el5/i386
So with out further ado, lets get into the guts of actually getting everything pulled down, synced and a repository created.
Below is the script I use to keep my mirror in sync with the closest mirror that offers rsync:
1 #!/bin/sh
2
3 rsync="rsync -avrt --bwlimit=796"
4
5 mirror=rsync://centos.mirrors.tds.net/CentOS
6
7 verlist="5 5.5 5.6 5.7"
8 archlist="i386"
9 baselist="os updates cr"
10 local=/Storage/yumRepo/centos
11
12 for ver in $verlist
13 do
14 for arch in $archlist
15 do
16 for base in $baselist
17 do
18 remote=$mirror/$ver/$base/$arch/
19 $rsync $remote $local/$ver/$base/$arch/
20 createrepo -v --update $local/$ver/$base/$arch
21 done
22 done
23 done
24 hardlink /Storage/yumRepo/centos
Now, lets break this script down line by line. Line one is obvious, it tells the program loader what interpreter to use when running this file. Line three sets up the rsync command I'll be using. In my case I need to limit the bandwidth usage to something sane, even though I run it off peak hours. Line five sets the mirror I'm running against. Line seven sets the versions of CentOS I'm interested in. I really could trim this back to just 5 and 5.7 now, but I'm going to leave it in there for now. Line eight tells me I'm only going to sync i386 architect and not the x64 architect. I'm doing this because as of right now we only run x86. Line nine gives me which repositories in the i386 architect I'm going to mirror. Line ten sets up where on the local server I'm going to store everything.
Lines twelve through twenty three run rsync against all of the variations of repository and architect type for each version. In my case it would be twelve different syncs. If I where also running x64 architect it would double to twenty four. Line nineteen is the actual rsync command while line twenty creates the necessary repository files for yum to use.
Line twenty four is rather new to this script and requires you have the hardlink command installed (yum install hardlink). The hardlink command will run through everyfile in the directory specified and compare it to every other file in the directory. It is looking for the same contents with the same permissions, but it can have a different filename. If it finds a match it will hardlink one of them to the other, thus reducing the space required to store two copies. My only wish is that I had some hard stats for how much disk space this is saving me on the repository alone.
Now create the repository file for Yum.
#vi /etc/yum.repos.d/MyRepo.repo
This file contains:
[base]
name=CentOS-$releasever - Base
baseurl=repo.mydomain.com/centos/$releasever/os/$basearch/
gpgcheck=1
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-5
protect=1
enabled=1
[update]
name=CentOS-$releasever - Updates
baseurl=http://repo.mydomain.com/centos/$releasever/updates/$basearch/
gpgcheck=1
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-5
protect=1
enabled=1
[CompanyName]
name=CentOS-$releasever - Local
baseurl=http://repo.mydomain.com/local/el$releasever/$basearch
enabled=1
gpgcheck=0
protect=0
enabled=0
We need to also tell Apache about these directories and how to serve them out over HTTP.
I changed the DocumentRoot to
DocumentRoot "/Storage/yumRepo"
then I changed the Default directory to
Options +Indexes +FollowSymLinks
AllowOverride All
Allow from all
Restart apache.
#service httpd restart
Populate Company Local repo with custom RPMs.
On every server we have installed Webmin. In the past this was installed either by hand or via our install script, but that had to be updated by hand with each update. So I will move the latest into our Company Local repo.
#mv /Storage/rpmDownloads/webmin-1.510.rpm /Storage/yumRepo/local/el5/i386/
You can move as many files into here as you want, including built from source RPMs that you may have made. After you move in any custom RPMs you have to update the Repository.
#createrepo -v --update /Storage/yumRepo/local/el5/i386/
First off create the directories that will become the repositories.
mkdir -pv /Storage/yumRepo/{5,5.4,5.5/{os,updates,cr}/i386
mkdir -pv /Storage/yumRepo/local/el5/i386
So with out further ado, lets get into the guts of actually getting everything pulled down, synced and a repository created.
Below is the script I use to keep my mirror in sync with the closest mirror that offers rsync:
1 #!/bin/sh
2
3 rsync="rsync -avrt --bwlimit=796"
4
5 mirror=rsync://centos.mirrors.tds.net/CentOS
6
7 verlist="5 5.5 5.6 5.7"
8 archlist="i386"
9 baselist="os updates cr"
10 local=/Storage/yumRepo/centos
11
12 for ver in $verlist
13 do
14 for arch in $archlist
15 do
16 for base in $baselist
17 do
18 remote=$mirror/$ver/$base/$arch/
19 $rsync $remote $local/$ver/$base/$arch/
20 createrepo -v --update $local/$ver/$base/$arch
21 done
22 done
23 done
24 hardlink /Storage/yumRepo/centos
Now, lets break this script down line by line. Line one is obvious, it tells the program loader what interpreter to use when running this file. Line three sets up the rsync command I'll be using. In my case I need to limit the bandwidth usage to something sane, even though I run it off peak hours. Line five sets the mirror I'm running against. Line seven sets the versions of CentOS I'm interested in. I really could trim this back to just 5 and 5.7 now, but I'm going to leave it in there for now. Line eight tells me I'm only going to sync i386 architect and not the x64 architect. I'm doing this because as of right now we only run x86. Line nine gives me which repositories in the i386 architect I'm going to mirror. Line ten sets up where on the local server I'm going to store everything.
Lines twelve through twenty three run rsync against all of the variations of repository and architect type for each version. In my case it would be twelve different syncs. If I where also running x64 architect it would double to twenty four. Line nineteen is the actual rsync command while line twenty creates the necessary repository files for yum to use.
Line twenty four is rather new to this script and requires you have the hardlink command installed (yum install hardlink). The hardlink command will run through everyfile in the directory specified and compare it to every other file in the directory. It is looking for the same contents with the same permissions, but it can have a different filename. If it finds a match it will hardlink one of them to the other, thus reducing the space required to store two copies. My only wish is that I had some hard stats for how much disk space this is saving me on the repository alone.
Now create the repository file for Yum.
#vi /etc/yum.repos.d/MyRepo.repo
This file contains:
[base]
name=CentOS-$releasever - Base
baseurl=repo.mydomain.com/centos/$releasever/os/$basearch/
gpgcheck=1
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-5
protect=1
enabled=1
[update]
name=CentOS-$releasever - Updates
baseurl=http://repo.mydomain.com/centos/$releasever/updates/$basearch/
gpgcheck=1
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-5
protect=1
enabled=1
[CompanyName]
name=CentOS-$releasever - Local
baseurl=http://repo.mydomain.com/local/el$releasever/$basearch
enabled=1
gpgcheck=0
protect=0
enabled=0
We need to also tell Apache about these directories and how to serve them out over HTTP.
I changed the DocumentRoot to
DocumentRoot "/Storage/yumRepo"
then I changed the Default directory to
Options +Indexes +FollowSymLinks
AllowOverride All
Allow from all
Restart apache.
#service httpd restart
Populate Company Local repo with custom RPMs.
On every server we have installed Webmin. In the past this was installed either by hand or via our install script, but that had to be updated by hand with each update. So I will move the latest into our Company Local repo.
#mv /Storage/rpmDownloads/webmin-1.510.rpm /Storage/yumRepo/local/el5/i386/
You can move as many files into here as you want, including built from source RPMs that you may have made. After you move in any custom RPMs you have to update the Repository.
#createrepo -v --update /Storage/yumRepo/local/el5/i386/
No comments:
Post a Comment