FreeNAS Development Environment

It isn't particularly easy to start developing on the FreeNAS platform (version 9), most of the OS being read-only is only the starting point of the kind of trouble you'll encounter if you are willing to engage in FreeNAS development/debugging/enhancement.

A document released during the 2014 FOSDEM event is still available on their archive page.

Also the README file at the root of the FreeNAS repo gives us instructions on how to build FreeNAS from source.

We'll try to setup such an environment inside a virtual machine, but we'll tune it up a little bit to fit our needs (mostly adding the VirtualBox Guest Additions).

OK, for the ones that do not want to take the deep dive into the gruesome details of building a FreeNAS development VM. Or if you simply don't have the time required to achieve the whole process, as it takes more than 3 hours to complete all described operations! Here is a download link to the .vdi file that is going to get build following the steps described in this article +/- 663MB (you lucky ones!):

Note that you'll probably have to create extra virtual drives for the VM to create a “virtual pool”.

Download FreeNAS-Dev-9.3.1.vdi

Create a New VM in VirtualBox

Name: FreeNAS-Builder
Type: BSD
Version: FreeBSD (64-bit)
Memory: 6144 MB
Processors: 4
Create virtual hard disk now
Disk file type: VDI
Dynamically allocated
32 GB

Configure network

As we usually do, we'll have a “development”, fixed, IP (to access the VM for management purposes) along with a “DHCP” IP (to access the Internet from the VM):

Network > Adapter 1 > NAT
Network > Adapter 2 > Host-only adapter / vboxnet0

Install FreeBSD 9.x

You can refer to the FreeBSD documentation on installing FreeBSD for more in-depth details.

Download the correct ISO file from the FreeBSD releases download page. We will use the FreeBSD-9.3-RELEASE-amd64-bootonly.iso version here.

Link the downloaded .iso file to your newly created VM in:

Storage > IDE Controller > Add optical drive > FreeBSD-9.3-RELEASE-amd64-bootonly.iso

Start the VM, select option 1 (default), or wait a few seconds for the FreeBSD installer to load.

Install
Choose your keymap (here French ISO-8859-1 (accent keys)
Set hostname (fnbuilder.dev)
Optional components → unselect “games”
Configure the “DHCP” created interface (em0) → use DHCP → DNS 8.8.8.8 / 8.8.4.4
NO ipv6
Resolver Configuration: Search : - DNS#1 : 8.8.8.8 - DNS#2 : 8.8.4.4
Select ftp mirror close to your location
Use GUIDED partitioning (whole disk)
Finish - Commit
Set root password
Set UTC > Choose your location
Services: sshd + ntpd + dumpdev
Finish the installation

Shutdown the VM and unlink the ISO file.

Install nano to edit text files

OK, let's admit it, I'm not a great fan of vi. If you know how to use it or would like to learn it, don't bother installing nano and use vi to edit text files. But this is also the occasion to illustrate how to add new software to FreeBSD (using ports):

> portsnap fetch update
> portsnap fetch extract update
> cd /usr/ports/editors/nano
> make install clean
SELECT ALL DEFAULTS OPTIONS
> ln -s /usr/local/bin/nano /usr/bin/nano

YEP!
This seems to be what it's like installing packages using portsnap… I must say that Debian's apt-get install nano has my preference

To be honest, you could simply issue:

> pkg install editors/nano

And the process would achieve as well

Setup dev network interface

> nano /etc/rc.conf

ADD:
# dev network interface
ifconfig_em1="inet 172.20.20.3 netmask 255.255.255.0"
defaultrouter="172.20.20.1"

Allow root SSH Login

> nano /etc/ssh/sshd_config

CHANGE:
#PermitRootLogin no
TO:
PermitRootLogin yes

Install ranger

Ranger is a useful console file manager with. It provides a minimalist and nice curses interface with a view on the directory hierarchy (and much more…).

> pkg install sysutils/py-ranger

Restart your VM.

You should now be able to access your VM via ssh from your host system using:

> ssh root@172.20.20.3

The main FreeNAS project is hosted at: http://github.com/freenas/freenas.git

We will build the FreeNAS installer from the make command. This will build the entire .iso installer for FreeNAS, but be advised that this will require a lot of files to be downloaded as a full checkout of TrueOS source and ports tree will take place.

TrueOS is FreeBSD with some FreeNAS specific modifications in it. A frozen ports tree is also used so versions of the ports they use don't get bumped without them knowing !

Install git, Python and other complementary packages

As root:

> pkg install devel/git archivers/pxz lang/python ports-mgmt/poudriere-devel sysutils/grub2-pcbsd sysutils/xorriso sysutils/cdrtools py27-sphinx py27-sphinxcontrib-httpdomain-1.2.1

Get the Sources from GitHub and Change build parameters

> cd /root/
> git clone http://github.com/freenas/freenas.git
[ > git clone -b 9.2.1-BRANCH http://github.com/freenas/freenas.git]

Refer to the GitHub page of the FreeNAS project to see available branches.

Build with portsnap and Ports

For security reasons, the standard installation of FreeNAS is pretty much closed and does not include portsnap, although we want to be able to install ports later on, so we require some possibility to install new packages onto the FreeNAS OS.

We will need to modify /root/freenas/build/nano_env as follow:

line 224, 225: comment out
#unset PACKAGEROOT
#unset PACKAGESITE

line 307 change "yes" to "no"
: ${WITH_PKGNG:="no"}

line 326: comment out
#WITHOUT_FREEBSD_UPDATE=yes

line 334: comment out
#WITHOUT_LIB32=true

line 344: comment out
#WITHOUT_PORTSNAP=true

line 346: comment out
#WITHOUT_PROFILE=true

line 352: comment out
#WITHOUT_SYSINSTALL=true

line 360: comment out
#NOPORTDOCS=true

lines 398, 399: comment out
#WITHOUT_GCC=true
#WITHOUT_INSTALLLIB=true

lines 416, 417: comment out
#WITHOUT_GCC=true
#WITHOUT_INSTALLLIB=true

It turns out that, as of this writing, the FreeBSD VirtualBox Guest Additions DO NOT YET SUPPORT SHARED FOLDERS!

See the FreeBSD VirtualBox ToDo page, where it is stated:
Guest Additions: Implement vboxsf (shared folders)

This means there is currently NO WAY of accessing a host's shared folder from a FreeBSD guest VM

It turns out that, although the above modifications permits ports installation on the FreeNAS system, installation of the emulators/virtualbox-ose-additions gets stuck for different reasons that we won't detail here, we'll try to have this, and a few others, installed at compile time. Therefore we'll modify /root/freenas/build/nanobsd-cfg/os-ports as follow:

Add at end of file:

add_port emulators/virtualbox-ose-additions PORT_OPTIONS=NONE
add_port sysutils/py-ranger

Finally, modify virtualbox-ose-additions to accept the PORT_OPTIONS=NONE value in /root/freenas/FreeBSD/ports/emulators/virtualbox-ose-additions/Makefile:

line 51: add NONE
OPTIONS_DEFINE= DEBUG OPENGL X11 NONE

/root/freenas/

/root/freenas/build/ports/install-ports.sh line 35:

Compile FreeNAS Installer iso

v 9.3+

> cd /root/freenas
> make checkout  
> make release
> make update

The install process for earlier versions of FreeNAS (9.2.x) is different and causes a LOT of complications so we won't cover it here.

If the process succeeds (took about 3 hours on the VM), the resulting .iso file should be available at:
/root/freenas/objs/FreeNAS-9.3.1-MASTER-2015xxxxxxxx/x64/FreeNAS-9.3-Nightlies-2015xxxxxxxx.iso

Transfer .iso file to host

On your host machine:

> cd /path/to/iso/repo/
> sftp root@172.20.20.3
> get /root/freenas/objs/FreeNAS-9.3.1-MASTER-2015xxxxxxxx/x64/FreeNAS-9.3-Nightlies-2015xxxxxxxx.iso

Prepare src for transfer to the FreeNAS VM [deprecated]

/usr/src files are required to install the virtualbox-ose-additions, by default they are NOT copied to the FreeNAS installation, so we'll prepare them for later recovery in the FreeNAS VM:

> cd /root/freenas-9.3.1
> tar -czf src.tgz objs/os-base/amd64/root/freenas-9.3.1/FreeBSD/src

From your host machine:

> cd /where/to/save/path/
> sftp root@172.20.20.3
> cd /root/freenas-9.3.1
> get src.tgz

We'll create a new FreeBSD 64-bit VM, booting from our built .iso image:

Name: FreeNAS-Dev-9.3.N
Type: BSD
Version: FreeBSD (64-bit)
Memory: 2048 MB
Processors: 1
Create virtual hard disk now
Disk file type: VDI
Dynamically allocated
8 GB

Network > Adapter 1 > NAT
Network > Adapter 2 > Host-only adapter / vboxnet0

Storage > Controller: IDE > Optical Drive > load FreeNAS-xxx.iso

You might get an error of type:
ACPI Exception: AE_ALREADY_EXISTS, Unable to install System Control Interrupt handler
If this happens, close your VM, and make sure to check: System > Motherboard > Enable I/O APIC

FreeNAS Installation

Once the VM has booted, installation panels will let you make selections:

Install/Upgrade
ada0 (your vbox hardisk)
Password…

Shutdown

VM Storage

Detach CD .iso image from VM.

As we'd like to experiment with zfs raids, we'll add a SATA controller and link 5 virtual HDs of 3GB each. Those .vdi files already exists from a previous install so we duplicate the .vdi files and change their UUID:

> cp /path/to/original/SATAHD*.vdi /path/to/new/
> vboxmanage internalcommands sethduuid /path/to/new/SATAHD1.vdi
> vboxmanage internalcommands sethduuid /path/to/new/SATAHD2.vdi
> vboxmanage internalcommands sethduuid /path/to/new/SATAHD3.vdi
> vboxmanage internalcommands sethduuid /path/to/new/SATAHD4.vdi
> vboxmanage internalcommands sethduuid /path/to/new/SATAHD5.vdi

Attach all disks to a new SATA interface on the VM and restart it…

Network interfaces

On the login screen, select 1) to configure the network interfaces. configure:

NIC: em0
Interface Name: NAT network
DHCP: YES
IPv6: NO

NIC: em1
Interface Name: vboxnet
DHCP: NO
IPv4 Address: 172.20.20.3
IPv4 Netmask: /24
IPv6: NO

You should now be able to access FreeNAS GUI through your browser at http://172.20.20.3

Basic setup

The first time we access the FreeNAS Web GUI, a wizard is presented, you can follow it. What has to be done in our case is first to set the correct keyboard mapping and timezone, this is done in the System > General panel.

We also need to activate the required services we want, we are activating the following one in the Services panel:

SSH - Login as root with password

Our VM can now be accessed through ssh from a shell terminal.

To enabled Internet access from our VM, we configure the second interface, go to:

Network > Interfaces > Add Interface
em0 > DHCP

Network > Global Configuration
Hostname: freenasDEV
Default Gateway : 10.0.2.1
Nameserver 1 : 8.8.8.8
Nameserver 2 : 8.8.4.4

VM Snapshot

As our FreeNAS dev platform is now basicaly configured, it's time to make a snapshot of it (using VirtualBox GUI application for example).

Additional Packages

portsnap

Let's first check that portsnap is installed and working:

> portsnap fetch update
> portsnap fetch extract update

VirtualBox Guest Additions

As stated earlier, and as of this writing, FreeBSD VirtualBox Guest Additions do NOT yet support shared folders!

We specifically changed the build options (above) to already include the VirtualBox Guest Additions. Although as described on FreeBSD's VirtualBox section, we still need to modify /etc/rc.conf as follow:

> nano /etc/rc.conf
Add at end of file:
# VirtualBox additions
vboxguest_enable="YES"
vboxservice_enable="YES"
vboxservice_flags="--disable-timesync"

Reboot the VM.

Using SSHFS to Access VM's Files

As it turns out FreeBSD's VB Guest Additions do not support Shared Folders, and SAMBA sharing only brings half a solution to our need of editing FreeNAS files from the host, development, workstation.

What we usually have are files residing on our host's FS, accessed by our VM through a sharing mechanism (Shared Folder or SAMBA mount). The coming solution will reverse that paradigm, letting the host workstation access VM's files as if they were on a regular FS. Note that in this scenario DESTROYING THE VM WILL DESTROY DEVELOPMENT DATA!

A detailed article is available in the Digital Ocean's tutorials in case one wants to use the same principle on OSX or Windows.

Prepare host workstation

On the host system:

> sudo apt-get install sshfs

Create a local directory in which to mount the VM's file system:

> sudo mkdir /mnt/FreeNAS-DEV-9.3.1
> sudo chown -R <devuser>:<devgroup> /mnt/FreeNAS-9.3.1

(un)Mount the VM's FS on host system

> sshfs root@172.20.20.3:/ /mnt/FreeNAS-DEV-9.3.1

> umount /mnt/FreeNAS-DEV-9.3.1

Once again this configuration will leave all development files in the VM's filesystem. One might want to set-up some sort of automated folder sync to keep a copy on the host filesystem.

Create a SAMBA Share to Ease FreeNAS GUI's Files Edition

Although this configuration works for the /usr/local/www/freenasUI files, it WON'T WORK for the /data/freenas-v1.db database that is accessed during the boot process before SAMBA mounts are available, resulting in an unbootable system !

Another approach using sshfs mounting from the workstation to the VM might well be the easiest way to access/edit files on the FreeNAS development VM…

CLICK HERE TO READ

Since we won't be able to use shared folders with FreeBSD (as described in this wiki's Python/Django Development Environment section) we'll have to create a SAMBA shared folder to easily manage and edit FreeNAS GUI's files from our host system.

As the intention is to analyze, and maybe modify, some code in the FreeNAS GUI and/or api, our main target directory location is /usr/local/www/freenasUI. We'll first duplicate it's content to the host's HD, then make this duplicate directory a SAMBA shared folder that we'll access from the FreeNAS development VM. Finally this SAMBA share will be mounted, with adequate ownership and permissions, as the /usr/local/www/freenasUI directory inside the FreeNAS development VM.

We'll do the same to gain easy access to the FreeNAS database (in /data/freenas-v1.db)

Duplicate freenasUI directory and make SAMBA share

From the host machine command line:

> cd /host/path/to/shared-folders/
> sftp root@172.20.20.3
> cd /usr/local/www/
> get -r freenasUI
> lcd /host/path/to/shared-folders/data/
> cd /data
> get freenas-v1.db
> exit

So we now have two folders in /host/path/to/shared-folders/ :

data
freenasUI

The steps to actually share those two folders from your host using SAMBA depends on your host's OS and Desktop Environment. We won't cover the specifics here but google is your friend

On my system (LMDE 2/Cinnamon), all that was required was:

right click on the folders' icons
Sharing Options > check “Share this folder” > Enter “Share name” > check both “Allow others to create and delete files” and “Guest access”.

Our shares names are: fnui-UI and fnui-data.

Test mount your SAMBA shares

Let's create a temporary mount point and make sure our shares can be accessed (in the guest VM).
The global format of the command is: mount_smbfs -I <server-ip> / /<user-name>@<server-name>/<share-name> /local/mount/point:

> mkdir /tmp/fnui
> mount_smbfs -I <172.20.20.1> //john@lmde2-w150/fnui-UI /tmp/fnui
Password:
> ls /tmp/fnui
./                  common/             local_settings.pyc* settings.py*        templates/
../                 contrib/            locale/             settings.pyc*       tools/
__init__.py*        directoryservice/   manage.py*          sharing/            urls.py*
__init__.pyc*       documentation/      middleware/         static/             urls.pyc*
account/            dojango/            network/            storage/
api/                fnstatic/           plugins/            support/
choices.py*         freeadmin/          reporting/          system/
choices.pyc*        jails/              services/           tasks/
> umount /tmp/fnui

YEP!
We are now listing our host's folders content inside our VM. We still need to configure some things before we get running though.

Configure Auto-mounting of the SAMBA Shares

What we want, ultimately, is to have our fnui-UI share mounted to /usr/local/www/freenasUI, with the correct permissions for the system to run. Also our fnui-data/freenas-v1.db should “replace” /data/freenas-v1.db…

Inspect owners and permissions

From the FreeNAS guest VM command line:

> ls -la /usr/local/www/ | grep freenasUI
drwxr-xr-x  26 www   www    37 Aug 17 20:35 freenasUI/
> id www
uid=80(www) gid=80(www) groups=80(www)

> ls -la /data/ | grep freenas
-rw-r-----   1 root  operator  319488 Aug 21 03:43 freenas-v1.db
> id operator
uid=2(operator) gid=5(operator) groups=5(operator)

Test mount options

In the above mount test, one had to introduce a password to complete the mount_smbfs command (even if the password is blank).
As we intend to have the smb shares automatically mounted, we need to store the password in ~/.nsmbrc for our tests and in /etc/nsmb.conf for fstab.

Also, as we want the shared folder to be mounted with the correct permissions we'll add some options to the mount_smbfs command:

> nano ~/.nsmbrc
Write:
[SERVERNAME:USERNAME]
password=myPassword

> mount_smbfs -N -u 80 -g 80 -f 755 -d 755 -I 172.20.20.1 //FNGUEST@LMDE2-W150HR/fnui-UI /tmp/fnui
> ls -la /tmp/fnui
total 483
drwxr-xr-x  1 www   www    16384 Jan  1  1970 ./
drwxrwxrwt  6 root  wheel    360 Aug 21 16:51 ../
-rw-r--r--  1 www   www     1365 Aug 17 23:29 __init__.py
-rw-r--r--  1 www   www      123 Aug 17 23:30 __init__.pyc
drwxr-xr-x  1 www   www    16384 Aug 17 23:30 account/
drwxr-xr-x  1 www   www    16384 Aug 17 23:30 api/
-rw-r--r--  1 www   www    28484 Aug 17 23:30 choices.py
-rw-r--r--  1 www   www    32839 Aug 17 23:29 choices.pyc
drwxr-xr-x  1 www   www    16384 Aug 17 23:30 common/
...
> umount /tmp/fnui

Note:
Both the hostname and the username need to be in UPPERCASE.
The password can be empty.
We now use the -N option for mount_smbfs to force password lookup in ~/.nsmbrc.
We also use the -u (user-id), -g (group-id), -f (files permissions) and -d (directories permissions) options to get the correct ownership and rights on the mounted files.
The uncommon -f 755 usage comes from the fact that python files need execution rights.

Automount using fstab

Now that we are confident the mount process is working, let's automate it using /etc/fstab.

First, fstab will look for passwords in /etc/nsmb.conf so:

> nano /etc/nsmb.conf

Add at the end of file:
# VBox Shared Folders
[SERVERNAME:USERNAME]
password=thePassword

Next, our fnui-UI shared will be mounted at /usr/local/www/freenasUI, that is pretty straight forward.
Regarding our fnui-data share, things are a little different, we need only one file (freenas-v1.db) “replaced” by our host located file. To achieve this, we'll create a specific mount point for fnui-data, then symlink the “original” file pointing to our mounted location.

Finally, the fstab syntax is slightly different from the command line, see how it is structured in the following code snippet:

> mkdir /mnt/shared-data
> nano /etc/fstab

Add at end of file:
# VBox smb shared folders
//FNGUEST@LMDE2-W150HR/fnui-UI /usr/local/www/freenasUI smbfs rw,-N,-u80,-g80,-f755,-d755,-I172.20.20.1 0 0
//FNGUEST@LMDE2-W150HR/fnui-data /mnt/shared-data smbfs rw,-N,-u0,-g5,-f640,-d755,-I172.20.20.1 0 0

> mount -a

At this point you should have your shared freenasUI files replacing the /usr/local/www/freenasUI directory and the freenas-v1.db in /mnt/shared-data.

Let's create the necessary symlink for the FreeNAS database and copy the local_settings.py file:

> mv /data/freenas-v1.db /data/freenas-v1.db.bak
> ln -s /mnt/shared-data/freenas-v1.db /data/freenas-v1.db
> cp /etc/local_settings.py /usr/local/www/freenasUI/

Let's restart our VM to check everything goes as expected once we reboot the machine.

Now that we have access to the FreeNAS files, we'll create an IntelliJ IDEA project to manage our development actions.

Open IntelliJ IDEA and create a new project:

Project type: Python
Project SDK:
- … → Add Remote
- SSH Credentials
- Host: 172.20.20.3
- User name: root
- Password: 123456
- Python interpreter path: /usr/local/bin/python
Additional Libraries and Frameworks: Django
Template language: Django
Templates folder: templates
Application name: FreeNAS-9.3.1
No Django admin

→ Next → Next

Project name: FreeNAS-9.3.1
Project location: /mnt/FreeNAS-DEV-9.3.1/usr/local/www/freenasUI
Leave all default for “More Settings”

→ Finnish

Warning says files cannot be watched for external update because they are network-mounted.

Initiate git repo

(menu) VCS > Enable Version Control Integration : git