Tag: 185

  • Building DVD Images Of Ubuntu Repositories

    1 Preliminary Note

    This tutorial was inspired by an articles I read at http://cargol.net/~ramon/ubuntu-dvd-en. So many thanks to Ramon Acedo (the one who made this HowTo, originally)

    The pages are not reachable from some weeks, now. I saved the page to read it off-line. So…

    I found it useful. I hope it will be the same for you.

    2 Introduction

    This howto offers a simple way of creating DVD images of Debian or Ubuntu http/ftp repositories.

    Ubuntu doesn’t offer DVDs ready to download with its main, universe, multiverse and/or restricted repositories. With the contents of this howto you can do it yourself.

    Having the Ubuntu or Debian repositories on DVD can be useful for those users who don’t have access to the Internet where they have their Ubuntu installed but have access somewhere else to download the repository and build and burn the DVDs.

    3 Building a local mirror

    We have to install debmirror:

    sudo apt-get install debmirror

    Now we get the Ubuntu repositories in a local directory. In the example below we get main, universe and multiverse sections of the repository in the i386 architecture.

    debmirror –nosource -m –passive –host=archive.ubuntulinux.org –root=ubuntu/ –method=ftp –progress –dist=dapper –section=main,multiverse,universe –arch=i386 ubuntu/ –ignore-release-gpg

    You could change the options below as you prefer:

    • –host – the URL of the repository.
    • –dist – the distro of your OS (dapper, edgy, sarge, … ).
    • –section – the section you want to mirror locally.
    • –arch – the architecture of your box.

    4 Separating the archive into DVD-sized directories

    The repositories we got are too big (about 30Gb) to burn them to a DVD so we have to separate them into volumes.

    The tool debpartial will do it for us.

    sudo apt-get install debpartial

    We make the directory where the volumes will reside.

    mkdir ubuntu-dvd

    and we make it to construct the package descriptors to every volume.

    debpartial –nosource –dirprefix=ubuntu –section=main,universe,multiverse –dist=dapper –size=DVD ubuntu/ ubuntu-dvd/

    Now we have to put the packages into the directories debpartial has just created. The script debcopy which also comes with the debpartial package will do it. The script needs ruby.

    sudo apt-get install ruby

    If everything is ok…

    ruby debcopy ubuntu/ ubuntu-dvd/ubuntu0
    ruby debcopy ubuntu/ ubuntu-dvd/ubuntu1
    ruby debcopy ubuntu/ ubuntu-dvd/ubuntu2

    Where ubuntu/ is the directory with the complete repository created with debmirror and ubuntu-dvd/* are the directories ready to host the new DVD-ready repository.
    If we want to make soft links from the complete repository instead of copying the packages we can call debcopy with the option -l:

    ruby -l debcopy ubuntu/ ubuntu-dvd/ubuntu0
    ruby -l debcopy ubuntu/ ubuntu-dvd/ubuntu1
    ruby -l debcopy ubuntu/ ubuntu-dvd/ubuntu2

    Now every directory (ubuntu0, ubuntu1 and ubuntu2) fits on one DVD.

    5 Making iso images

    To get the directories ubuntu0, ubuntu1, ubuntu2 into an iso image ready to burn we can use mkisofs:

    mkisofs -f -J -r -o ubuntu-dvd-0.iso ubuntu-dvd/ubuntu0
    mkisofs -f -J -r -o ubuntu-dvd-1.iso ubuntu-dvd/ubuntu1
    mkisofs -f -J -r -o ubuntu-dvd-2.iso ubuntu-dvd/ubuntu2

    Now you can burn the iso images or mount them. Add them to the /etc/apt/source.list with the command:

    sudo apt-cdrom add

    Now we can verify the new repositories…

    sudo apt-get update
    sudo apt-get upgrade

    … and, if I explain in the right way, you should have your box upgraded.

    6 About the script ‘debcopy’

    I heard about someone who can not find the script debcopy, above described.
    In that case, create a new file called debcopy where you want:

    gedit /your_path_to/debcopy

    and copy the lines below inside it:

    #!/usr/bin/ruby
    #
    # debcopy - Debian Packages/Sources partial copy tool
    #
    # Usage: debcopy [-l]  
    #
    #  where  is a top directory of a debian archive,
    #  and  is a top directory of a new debian partial archive.
    #
    #  debcopy searches all Packages.gz and Sources.gz under /dists
    #  and copies all files listed in the Packages.gz and Sources.gz
    #  files into  from . -l creates symbolic links
    #  instead of copying files.
    #
    # Copyright (C) 2002  Masato Taruishi 
    #
    #  This program is free software; you can redistribute it and/or modify
    #  it under the terms of the GNU General Public License as published by
    #  the Free Software Foundation; either version 2 of the License, or
    #  (at your option) any later version.
    #
    #  This program is distributed in the hope that it will be useful,
    #  but WITHOUT ANY WARRANTY; without even the implied warranty of
    #  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    #  GNU General Public License for more details.
    #
    #  You should have received a copy of the GNU General Public License with
    #  the Debian GNU/Linux distribution in file /usr/share/common-licenses/GPL;
    #  if not, write to the Free Software Foundation, Inc., 59 Temple Place,
    #  Suite 330, Boston, MA  02111-1307  USA
    #
    require 'getoptlong'
    require 'zlib'
    require 'ftools'
    $link = false
    def usage
      $stderr.puts "Usage: #{__FILE__} [-l]  "
       exit 1
    end
    def each (file, &block)
      fin = Zlib::GzipReader.open(file)
      fin.each do |line|
        yield line
      end
      fin.close
    end
    def each_file (file, &block)
      each(file) do |line|
        if /Filename: (.*)/ =~ line
          yield $1
        end
      end
    end
    def each_sourcefile (file, &block)
      dir = nil
      each(file) do |line|
        case line
        when /^Directory: (.*)$/
          dir = $1
        when /^ \S+ \d+ (\S+)$/
          yield dir + "/" + $1
        end
      end
    end
    def calc_relpath (source, dest)
      pwd = Dir::pwd
      Dir::chdir source
      source = Dir::pwd
      Dir::chdir pwd
      Dir::chdir dest
      dest = Dir::pwd
      Dir::chdir pwd
      src_ary = source.split("/")
      src_ary.shift
      dest_ary = dest.split("/")
      dest_ary.shift
      return dest if src_ary[0] != dest_ary[0]
      src_ary.clone.each_index do |i|
        break if src_ary[0] != dest_ary[0]
        src_ary.shift
        dest_ary.shift
      end
      src_ary.size.times do |i|
        dest_ary.unshift("..")
      end
      dest_ary.join("/")
    end
    def do_copy(path)
      if $link
        pwd=calc_relpath(File.dirname($dest_dir + "/" + path), $source_dir)
        File.symlink(pwd + "/" + path, $dest_dir + "/" + path)
      else
        File.copy($source_dir + "/" + path, $dest_dir + "/" + path)
      end
    end
    def copy(path)
      s=$source_dir + "/" + path
      d=$dest_dir + "/" + path
      if FileTest.exist?(d)
        $stats["ignore"] += 1
        return
      end
      if FileTest.exist?(s)
        File.mkpath(File.dirname(d))
        do_copy(path)
        $stats["copy"] += 1
      else
        $stats["notfound"] += 1
        $stderr.puts s + " not found."
      end
    end
    opts = GetoptLong.new(["--symlink", "-l", GetoptLong::NO_ARGUMENT],
    		      ["--help", "-h", GetoptLong::NO_ARGUMENT])
    opts.each do |opt,arg|
      case opt
      when "--symlink"
        $link = true
      when "--help"
        usage
      end
    end
    usage if ARGV.size != 2
    $source_dir = ARGV.shift
    $dest_dir = ARGV.shift
    if $link
      $source_dir = Dir::pwd + "/" + $source_dir unless $source_dir =~ /\A\//
      $dest_dir = Dir::pwd + "/" + $dest_dir unless $dest_dir =~ /\A\//
    end
    $stats = {}
    $stats["ignore"] = 0
    $stats["copy"] = 0
    $stats["notfound"] = 0
    open("|find #{$dest_dir}/dists -name Packages.gz") do |o|
      o.each_line do |file|
        file.chomp!
        print "Processing #{file}... "
        $stdout.flush
        each_file(file) do |path|
          copy(path)
        end
        puts "done"
      end
    end
    open("|find #{$dest_dir}/dists -name Sources.gz") do |o|
      o.each_line do |file|
        file.chomp!
        print "Processing #{file}... "
        $stdout.flush
        each_sourcefile(file.chomp) do |path|
          copy(path)
        end
        puts "done"
      end
    end
    puts "Number of Copied Files: " + $stats["copy"].to_s
    puts "Number of Ignored Files: " + $stats["ignore"].to_s
    puts "Number of Non-existence File: " + $stats["notfound"].to_s