Novell Home

Ocean howto build

From Developer Community

The following covers setting up an entire build environment from your base operating system, any other configuration is not considered because I would need a lot more tea.


Contents

Install VMware Server

Head over to the VMware website, download the VMware Server and signup to receive a valid serial number. Install on your preferred machine following the provided instructions.



Create FreeBSD Virtual Machine

Start the VMware Server Console, login and create a new virtual machine.

Connect to Host
Enlarge
Connect to Host
VMware Server Console
Enlarge
VMware Server Console

Follow the wizard steps.

New Virtual Machine Wizard
Enlarge
New Virtual Machine Wizard
Virtual Machine Configuration: Select Typical
Enlarge
Virtual Machine Configuration: Select Typical
Guest Operating System: Select Other and FreeBSD
Enlarge
Guest Operating System: Select Other and FreeBSD
Name Virtual Machine: Choose an appropriate name for this machine
Enlarge
Name Virtual Machine: Choose an appropriate name for this machine
Network Type: Choose bridged networking
Enlarge
Network Type: Choose bridged networking
Disk Size: Leave on defaults, you can deselect "Allocate all disk space now" to save space, and deselect "Split disk into 2GB files" if your operating system supports files larger than 2GB
Enlarge
Disk Size: Leave on defaults, you can deselect "Allocate all disk space now" to save space, and deselect "Split disk into 2GB files" if your operating system supports files larger than 2GB
FreeBSD Developer
Enlarge
FreeBSD Developer



Install FreeBSD

This is a walkthrough of an example minimal install, please feel free to ignore and use the better FreeBSD Handbook instead.

Acquire the FreeBSD 6.1 disc 1 ISO and configure VMware to use this in the CD device. Power on the virtual machine and start the FreeBSD installation process.

Virtual machine BIOS splash screen
Enlarge
Virtual machine BIOS splash screen
Stage 0 through 2 of the boot process
Enlarge
Stage 0 through 2 of the boot process
FreeBSD Boot loader prompt
Enlarge
FreeBSD Boot loader prompt
First screen of installation program:  Country Selection
Enlarge
First screen of installation program: Country Selection

Select a Standard installation.

FreeBSD sysinstall main menu
Enlarge
FreeBSD sysinstall main menu

Create one partition that uses the entire disk which should be the default size, type should be BSD slice "165". Set the partition to be Bootable (S).

FreeBSD FDISK Partition Editor with one partition using all available disk space
Enlarge
FreeBSD FDISK Partition Editor with one partition using all available disk space

Install a Boot Manager.

FreeBSD Install Boot Manager for drive ad0?
Enlarge
FreeBSD Install Boot Manager for drive ad0?

BSD a bit different to Linux and Windows uses slices within a partition, the next step envolves configuration of these slices using the Disklabel Editor. Create a 256MB slice for swap and a FS slice for the remaining space mounted as "/" aka root. Enable SoftUpdates on the FS slice (S).

FreeBSD Disklabel Editor with two slices configured
Enlarge
FreeBSD Disklabel Editor with two slices configured

Select the Developer Distribution so we can get the compiler toolchain to rebuild FreeBSD and its packages. Confirm that you also want to install the Ports system.

FreeBSD Choose Distributions with Developer selected
Enlarge
FreeBSD Choose Distributions with Developer selected

If you have the full FreeBSD installation disk continue installing from the CD or DVD, otherwise configure network settings to retrieve the files from a local file server or mirror of the FreeBSD installation.

Wait for the installation to finish.

FreeBSD Extraction progress bar
Enlarge
FreeBSD Extraction progress bar

Now configure the network interfaces, for VMware this is the "lnc0 Lance/PCnet device".

FreeBSD Network interface information required
Enlarge
FreeBSD Network interface information required

Answer the questions appropriate for how you wish to use and access the machine. I answer as following:

  1. Do you want this machine to function as a network gateway? No
  2. Do you want to configure inetd and the network services that it provides? No
  3. Would you like to enable SSH login? Yes
  4. Do you want to have anonymous FTP access to this machine? No
  5. Do you want to configure this machine as an NFS server? No
  6. Do you want to configure this machine as an NFS client? Yes
  7. Would you like to customize your system console settings? Yes > Saver > Timeout = 0
  8. Would you like to set this machine's time zone now? Yes
  9. Is this machine's CMOS clock set to UTC? Yes
  10. Would you like to enable Linux binary compatibility? No
  11. Does this system have a PS/2, serial, or bus mouse? No
  12. Would you like to browse the collection now? No
  13. Would you like to add any initial user accounts to the system? Yes > User > Member Groups: wheel
  14. Visit the general configuration menu for a chance to set any last options? No

➊ Access the virtual machine remotely for a larger console and the ability to copy & paste.

➋ I use a NFS server to copy my releases.

➌ Disable terminal screen saver.

➍ Add your own user name and add membership to the wheel group, this allows escalation to superuser priviledges.


Exit the installation, and the machine will reboot into your newly installed copy of FreeBSD. After the screen finishes scrolling with kernel messages you will be asked to type in some random data and hit enter and the SSH encryption keys will be generated, and finally a prompt to login will appear.

Congratulations FreeBSD is installed and working!

FreeBSD System console
Enlarge
FreeBSD System console



Setup FreeBSD

We need to add some basic packages and configuration before we can update FreeBSD. This will also install a lot of dependencies and make take some time, it would be faster to install from CD but more convenient on the command line. You can install multiple packages on one line but the cdrtools package does not like this :(

# pkg_add -r ccache
# pkg_add -r cvsup-without-gui
# pkg_add -r subversion
# pkg_add -r portupgrade
# pkg_add -r cdrtools

Configure CVSUP for upgrading FreeBSD, copy two configuration files and modify to point to your fastest mirror.

# cp /usr/share/examples/cvsup/stable-supfile /etc/supfile
# cp /usr/share/examples/cvsup/ports-supfile /etc/ports-supfile

Scroll down past the comments and find the following section, update the host and release appropriately.

# Defaults that apply to all the collections
#
# IMPORTANT: Change the next line to use one of the CVSup mirror sites
# listed at http://www.freebsd.org/doc/handbook/mirrors.html.
*default host=CHANGE_THIS.FreeBSD.org ➊
*default base=/var/db
*default prefix=/usr
# The following line is for 6-stable.  If you want 5-stable, 4-stable,
# 3-stable, or 2.2-stable, change to "RELENG_5", "RELENG_4", "RELENG_3",
# or "RELENG_2_2" respectively.
*default release=cvs tag=RELENG_6_1 ➋
*default delete use-rel-suffix

➊ The host format is generally "cvsup.<country code>.FreeBSD.org", e.g. "cvsup.jp.FreeBSD.org".

➋ Update the CVS TAG to RELENG_6_1 to get FreeBSD 6.1.


Create an /etc/make.conf file with flags appropriate for your host platform and configuration for the compiler cache. For example if you have an Intel Pentium 4 you can run the following (Ctrl+D to terminate). If you cannot copy & paste simply cat the howto file to the end of make.conf and edit out everything else.

# cat >> /etc/make.conf
CPUTYPE=pentium4 ➊
CFLAGS= -O2 -pipe ➋
# ccache configuration ➌
.if !defined(NOCCACHE)
.if ${.CURDIR:M/usr/src*} && exists(/usr/local/libexec/ccache/cc)
CC=/usr/local/libexec/ccache/cc
CXX=/usr/local/libexec/ccache/c++
.else
CC=cc
CXX=c++
.endif
.else
CC=/usr/bin/cc
CXX=/usr/bin/c++
.endif

➊ CPU type or architecture as understood by the GCC compiler, WRAP is i586, Soekris are often i486.

➋ Compiler flags specifying compiler optimisations, choice usually between "-Os" for optimise for size, "-O" optimise for speed, "-O2" extra speed optimisations.

➌ ccache configuration as per instructions listed in /usr/local/share/doc/ccache/ccache-howto-freebsd.txt


The login environment also needs to be updated to use ccache.

# cat >> /etc/profile
export PATH=/usr/local/libexec/ccache:$PATH
export CCACHE_PATH=/usr/bin:/usr/local/bin
# cat >> /etc/csh.cshrc
setenv PATH /usr/local/libexec/ccache:$PATH
setenv CCACHE_PATH /usr/bin:/usr/local/bin

You will need to logout and re-login for this to take effect.



Update FreeBSD (Optional)

Not necessary to get something working, but if you want the latest bug fixes and updates. This process takes a bit of time, the compiler cache will help future rebuilds though.

First we need to synchronise the latest files.

# cvsup -g -L 2 /etc/supfile
# cvsup -g -L 2 /etc/ports-supfile


Update World and Kernel

As the root user execute the following.

# cd /usr/src
# make -j4 buildworld ➊
# make buildkernel ➋
# make installkernel
# reboot

➊ The option "-j4" runs four processes in parallel, and is recommended for fast single processor machines.

➋ Both steps can be combined by running "make kernel".


The machine will reboot, at the boot loader prompt select single user mode (4) or from the boot loader prompt: "boot -s".

# fsck -p
# mount -u /
# mount -a -t ufs
# swapon -a
# adjkerntz -i
# mergemaster -p ➊
# cd /usr/src
# make installworld
# make delete-old
# mergemaster ➋
# reboot

➊ ➋ You might have to answer a lot of questions because of file changes at these stages, refer to mergemaster -p and mergemaster for some tips on answering the questions.


The machine will reboot again, login normally and finish the cleanup process.

# cd /usr/src
# make delete-old-libs

Now skip to updating the ports.


Re-update World

In order to rebuild the world from a clean slate you need to destroy the current object directory.

# chflags -R noschg /usr/obj ➊
# rm -rf /usr/obj
# cd /usr/src
# make clean
# make -j4 buildworld
# make buildkernel

➊ Removes ACLs from files that prevents removal even with superuser priviledges.


If this fails you can try removing the source complete and rsyncing upstream:

# rm -rf /usr/src
# cvsup -g -L 2 /etc/supfile


Update Ports

As the root user execute the following.

# cd /usr/ports
# make fetchindex ➊
# portversion -l "<" ➋
# portupgrade -arR

➊ Downloads an indexed directory of ports, you can build your own using "make index".

➋ Shows all the ports that the index indicates newer versions are available, you can skip the next step if none, or no directly important ones are listed.



Build Ocean

Download Ocean from Subversion

Following the Subversion instructions checkout a copy of Ocean via HTTPS or SSH.

# cd /usr/local ➊
# svn checkout https://forgesvn1.novell.com/svn/miru-ocean/trunk/ocean

➊ The directory is currently hardcoded in one script /usr/local/ocean/inc/common.inc.sh


Download FreeSBIE 2 from CVS

As tarballs are only available for version 1 we need to retrieve from CVS, a script has been setup for this purpose.

# cd /usr/local/ocean
# ./fetch_freesbi2.sh
Retrieving FreeSBIE 2 from CVS ...
Finished.

Patch FreeSBIE 2 for adding procfs to default mount table.

# cp patches/freesbie2_scripts_img.sh freesbie2/scripts/img.sh
# cp patches/freesbie2_scripts_iso.sh freesbie2/scripts/iso.sh


Download packages

Some special packages are required that must be downloaded manually.

# cd /usr/local/ocean/downloads
# fetch http://www.freenas.org/downloads/clog-1.0.1.tar.gz
# fetch http://www.freenas.org/downloads/syslogd_clog-current.tgz
# fetch http://www.acme.com/software/mini_httpd/mini_httpd-1.19.tar.gz
# fetch http://www.freenas.org/downloads/mini_httpd.c.patch

The latest PHP 4.x stable package available from http://www.php.net/downloads.php for example.

# fetch http://www.php.net/get/php-4.4.2.tar.bz2/from/www.php.net/mirror

➊ This will download as a file "mirror" simply rename to "php-4.4.2.tar.bz2"


The Samba 4 Technology Preview 2 which should be available from http://www.samba.org/samba/ftp/samba4/ for example.

# fetch http://www.samba.org/samba/ftp/samba4/samba-4.0.0tp2.tar.gz


Build dependent packages

The build directory contains a collection of scripts to extract and build each package. Version changes of downloaded packages will require script changes to match.

Easy method:

# cd /usr/local/ocean/build
# ./build_all.sh

The following sections detail each individual component.


CLOG

A custom circular log file manager and modification to stock syslog daemon.

# cd /usr/local/ocean/build
# ./extract_clog.sh
# ./build_clog.sh
# ./extract_syslogd_clog.sh
# ./build_syslogd_clog.sh


gettext

Libraries for gettext to implement i18n.

# ./build_libiconv.sh
# ./build_libintl.sh


mini_httpd

Basic web server that supports PHP scripting.

A patch set is required for the following features.

  • Adds a limit to the number of concurrent connections to prevent DoS attacks (since mini_httpd is a forking web server)
  • Adds captive portal mode support
  • Sets SCRIPT_FILENAME (for php)
  • Fixes a bug where mini_httpd would exit if a TCP connection was closed before mini_httpd had a chance to handle it (ECONNABORTED)
  • Adds "index.php" to the list of index file names
  • Makes the error pages much more plain
  • Removed Server: header
  • Modify the Makefile to build mini_httpd with SSL support


Build with the following commands.

# ./extract_mini_httpd.sh
# ./build_mini_httpd.sh


PHP

Scripting language. Stable version 4.x without MySQL, PEAR, or SSL.

# ./extract_php4.sh
# ./build_php4.sh


Samba

Technology Preview 2 of Microsoft SMB/CIFS protocol implementation with Active Directory login support and LDAP service.

This builds static modules, a script exists to build a dynamic module version but fails to build without significant intervention. The file sizes can be reduced by removing uneeded modules, but Samba 4 has a lot of modules and little documentation :(

# ./extract_samba4.sh
# ./build_samba4.sh


ATAidle

For HDD installations ataidle allows the hard disk to spin down on idle.

# ./build_ataidle.sh


beep

A utility to generate a system beep to indicate startup complete.

# ./build_beep.sh


DHCP daemon

The ISC Dynamic Host Configuration Protocol daemon.

First time compiling you will be presented with an options menu, match the following selections.

[X] DHCP_PARANOIA
[X] DHCP_JAIL
[ ] DHCP_SOCKETS
[ ] DHCP_LDAP
[ ] DHCP_LDAP_SSL
[ ] OPENSSL_BASE
[ ] OPENSSL_PORT
[ ] DHCP_LQ


Then run the build script.

# ./build_dhcpd.sh


NTP client

A network time protocol (NTP) client.

# ./build_msntp.sh


rsync

A file synchronisation tool.

First time compiling you will be presented with an options menu, match the following selections.

[ ] POPT_PORT
[ ] TIMELIMIT
[X] FLAGS
[X] ACLS
[ ] SSH


TFTP daemon

A TFTP daemon which serves files to PXE booting clients.

# ./build_tftpd.sh


Build FreeSBIE image

To create the ISO and IMG files we need to go through a multiple stage FreeSBIE process, each stage is separated to allow changes to be made or stage re-runs without having to redo the entire process.


Stage 1: Configure FreeSBIE

Generate the FreeSBIE configuration file and calculate the image sizes.

# ./configure_freesbie2.sh


Stage 2: Create image patch

Create a clone of a patch set to default FreeSBIE with the packages we built in the previous steps.

# ./clone_ocean.sh


Stage 3: Build universe

Build the actual world and kernel files we are going to use in the final image.

# ./build_freesbie2.sh ➊


➊ This can be broken down into separate world and kernel scripts.

# ./buildworld.sh
# ./buildkernel.sh


Stage 4: Install universe

Once the world and kernel are built we need to install into the FreeSBIE file system.

# ./install_freesbie2.sh


Stage 5: Build disk image

Create a hard disk or compact flash disk image, also creates FreeSBIE clone file system needed for ISO building.

# ./img_freesbie2.sh


Stage 6: Build ISO image

For a CD-ROM ISO image.

# ./iso_freesbie2.sh
Creating FreeSBIE iso ...

/usr/local/ocean/freesbie-clone/usr: write failed filesystem is full ➊
Jul 18 18:44:05 freebsddev kernel: pid 39090 (dd), uid 0 inumber 8 on /usr/local ➋
/ocean/freesbie-clone/usr: filesystem full
Finished.
# ls -ltr release
total 15442
drwxr-xr-x 7 root  wheel       512 Jul 18 00:40 .svn
-rw-r--r-- 1 root  wheel  15792128 Jul 18 18:44 FreeSBIE.iso

➊ Ignore this error message.

➋ Details of this message will vary.

Novell® Making IT Work As One

© 2008 Novell, Inc. All Rights Reserved.