FreeNAS – getting hacks to survive an upgrade Part 2


***WARNING***

THIS DOES NOT MEAN YOU GET TO SHUT YOUR BRAIN OFF.  UPGRADING FREENAS COULD STILL RENDER YOUR CHANGES NULL AND VOID.  AFTER ANY FREENAS UPGRADE, VERIFY THAT YOUR CHANGES ARE STILL IN EFFECT EVEN WITH THIS STICKY HACK IN PLACE.

Part 1 of trying to get hacks to FreeNAS to survive an upgrade was a failure.  But never fear, I went back to the drawing board and had great success tonight. Basically I changed my approach slightly,  Instead of trying to make changes to /conf/base/etc/rc.conf , what ended up working was just making the changes to /etc/rc.conf on every boot.

This will have the added benefit of working even if the FreeNAS developers change their naming scheme again for things in /conf/base/etc.

Write our hacks.sh script

1. I have OpenVPN and pf Firewall enabled, neither of which are stock to FreeNAS.  You’ll need to adjust this script to meet your needs.  Open up your favorite text editor and save your hacks.sh file somewhere on your data drive on the FreeNAS box.

First thing is lets search /etc/rc.conf to see if our hacks are in place.  If they are, we exit.

#!/bin/sh
if grep "pf_enable" /etc/rc.conf
then
echo "Hacks in Place"
else

Now, the else section is where we add our hacks back in to /etc/rc.conf when they are not there.  I’m just using a simple echo command to write the line I want to add to the shell and then am adding it to the end of /etc/rc.conf.  I’m adding a blank line before my hacks, some notes about the hacks I’m adding and the config changes.  You will need to change the paths to pf_rules, openvpn_configfile and openvpn_dir to match your setup.

echo "" >> /etc/rc.conf
echo "#Turn on PF Firewall" >> /etc/rc.conf
echo "pf_enable='YES'" >> /etc/rc.conf
echo "pf_rules='/mnt/Files/joe/hacks/pf.conf'" >> /etc/rc.conf
echo "gateway_enable='YES'" >> /etc/rc.conf
echo "" >> /etc/rc.conf
echo "#Turn on OpenVPN" >> /etc/rc.conf
echo "openvpn_enable='YES'" >> /etc/rc.conf
echo "openvpn_if='tun'" >> /etc/rc.conf
echo "openvpn_configfile='/mnt/Files/openvpn/openvpn.conf'" >> /etc/rc.conf
echo "openvpn_dir='/mnt/Files/openvpn'" >> /etc/rc.conf

Next we want to start the services that we have added as hacks to FreeNAS.  For me that is pf and openVPN.  Last thing is to close out the if statement with fi.

service pf start
service openvpn start
fi

Here’s the full script.

#!/bin/sh
if grep "pf_enable" /etc/rc.conf
then
echo "Hacks in Place"
else
echo "" >> /etc/rc.conf
echo "#Turn on PF Firewall" >> /etc/rc.conf
echo "pf_enable='YES'" >> /etc/rc.conf
echo "pf_rules='/mnt/Files/joe/hacks/pf.conf'" >> /etc/rc.conf
echo "gateway_enable='YES'" >> /etc/rc.conf
echo "" >> /etc/rc.conf
echo "#Turn on OpenVPN" >> /etc/rc.conf
echo "openvpn_enable='YES'" >> /etc/rc.conf
echo "openvpn_if='tun'" >> /etc/rc.conf
echo "openvpn_configfile='/mnt/Files/openvpn/openvpn.conf'" >> /etc/rc.conf
echo "openvpn_dir='/mnt/Files/openvpn'" >> /etc/rc.conf
service pf start
service openvpn start
fi

Save it and exit.

2. Now we need to make our hacks.sh script executable and make it owned by root for good measure.  Make sure to change the path below to match your setup.

chmod 700 /mnt/Files/joe/hacks/hacks.sh
chown root /mnt/Files/joe/hacks/hacks.sh

Set up the FreeNAS GUI to run hacks.sh as an init script.

3. Log into the FreeNAS gui and go to System > Init/Shutdown Scripts.  Then click add Init/Shutdown Script

Screen Shot 2014-03-09 at 10.30.47 PM

You want to select script in the first field.  Browse to your hacks.sh script and select it in the second field and select Post Init in the third field.

Screen Shot 2014-03-09 at 10.31.06 PM

Hit OK.

4. Reboot and watch the magic.

Verification

5. After you’ve rebooted, check the status of your hacks.

service openvpn status

You should get this if you are running openVPN

openvpn is running as pid 17535.

service pf status

You should get this if you are running pf.  Notice that is says enabled.

No ALTQ support in kernel
ALTQ related functions disabled
Status: Enabled for 5 days 00:16:00           Debug: Urgent

State Table                          Total             Rate
current entries                       21
searches                      1236116133         2855.0/s
inserts                            75627            0.2/s
removals                           75649            0.2/s
Counters
match                            2299524            5.3/s
bad-offset                             0            0.0/s
fragment                               0            0.0/s
short                                  0            0.0/s
normalize                              0            0.0/s
memory                                 0            0.0/s
bad-timestamp                          0            0.0/s
congestion                             0            0.0/s
ip-option                          28950            0.1/s
proto-cksum                            0            0.0/s
state-mismatch                     94319            0.2/s
state-insert                           0            0.0/s
state-limit                            0            0.0/s
src-limit                             15            0.0/s
synproxy                               0            0.0/s

Final Thoughts

I set this up and tested it in a VM that started out as FreeNAS 9.2.0.  After setting this all up, I did a system upgrade using the GUI and upgraded to 9.2.1.2.  After rebooting, I checked /etc/rc.conf and my changes were in fact still there.  I then checked to see if pf and openVPN were running and they were.  SUCCESS!!!

I hope you all enjoy this.  Let me know of all the great ways you find to use this.

Advertisements

Secure FreeNAS 9.2.1.2 with a Firewall


The recent NTP reflection incident I was a victim of woke me up to the need for securing my FreeNAS boxes from outside connections.  Luckily, FreeNAS 9.2.1.2 comes shipped with the kernel extension for pf, so getting it working is pretty easy.

If you are going to do this on your FreeNAS box, please also check out my guide on getting hacks to FreeNAS to survive system upgrades.

DISCLAIMER – I AM NOT A SECURITY EXPERT, FOLLOW THIS GUIDE AT YOUR OWN RISK

I’ve been told that pf does not play nice with vimage jails.  It tends to cause kernel panics.

Get the basics done

1. Mount the filesystem so we can make some changes.

su
enter your root password
mount -uw /

2. Figure out what interfaces you have active so we can add them to the firewall rules.

ifconfig

Your output should look something like this:

re0: flags=8943<up,broadcast,running,promisc,simplex,multicast> metric 0 mtu 1500
options=82099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
ether f4:6d:04:db:21:ba
inet 192.168.0.10 netmask 0xffffff00 broadcast 192.168.0.255
inet6 xxxx::xxxx:xxxx:xxxx:xxxx%re0 prefixlen 64 scopeid 0x6
inet xxx.xxx.xxx.xxx netmask 0xfffffff8 broadcast 50.241.46.71
nd6 options=23<performnud,accept_rtadv,auto_linklocal>
media: Ethernet autoselect (1000baseT )
status: active

ipfw0: flags=8801<up,simplex,multicast> metric 0 mtu 65536
nd6 options=9<performnud,ifdisabled>

lo0: flags=8049<up,loopback,running,multicast> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0xa
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<performnud,auto_linklocal>

bridge0: flags=8843<up,broadcast,running,simplex,multicast> metric 0 mtu 1500
ether 02:df:7f:1c:ff:00
nd6 options=1
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: epair0a flags=143<learning,discover,autoedge,autoptp>
ifmaxaddr 0 port 12 priority 128 path cost 2000
member: re0 flags=143<learning,discover,autoedge,autoptp>
ifmaxaddr 0 port 6 priority 128 path cost 20000

epair0a: flags=8943<up,broadcast,running,promisc,simplex,multicast> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:09:09:00:0c:0a
nd6 options=1
media: Ethernet 10Gbase-T (10Gbase-T )
status: active

tun0: flags=8051<up,pointopoint,running,multicast> metric 0 mtu 1500
options=80000<LINKSTATE>
inet 10.8.0.1 –> 10.8.0.2 netmask 0xffffffff nd6 options=1
Opened by PID 17528

What we want to note are all the interface names.  Mine are re0, ipfw0, lo0, bridge0, epair0a and tun0

Create the firewall rules

3. Create the pf.conf file and edit it to your needs.  We are going to put it on our data drive so future FreeNAS upgrades don’t wipe it out. Adjust the path to match your setup.

mkdir /mnt/Files/hacks
vi /mnt/Files/hacks/pf.conf

Side note, vi has lots of commands, we just need to know a few:

x will delete the character your cursor is over, i will insert, esc will exit insert mode and to save and quit we use :wq

Here are the rules I am running.

#change this to match your primary ethernet interface, re0 or em0 are common, but there are others
ext_if="re0"
vpn_if="tun0"
table <bruteforce> persist
#These are all of the other interfaces we discovered in step 2
set skip on lo0
set skip on bridge0
set skip on ipfw0
set skip on epair0a
set skip on tun0
set block-policy return
scrub in all
#change xxx.xxx.xxx.xxx to the external IP of your FreeNAS box
nat on $ext_if from 10.8.0.0/24 to any -> xxx.xxx.xxx.xxx
#Lock it down
block in all
block out all
#Allow VPN traffic
pass on tun0 keep state
block quick from <bruteforce>
#Allow traffic in for ssh
pass in on $ext_if proto tcp from any to any port 22 flags S/SA keep state (max-src-conn 10, max-src-conn-rate 5/5, overload <bruteforce> flush global)
#Allow traffic in for web - delete or comment out if you don't want web traffic
pass in on $ext_if proto tcp from any to any port 80 flags S/SA keep state
pass in on $ext_if proto tcp from any to any port 443 flags S/SA keep state
#Allow traffic in for OpenVPN
pass in on $ext_if proto udp from any to any port 1194 keep state label "openvpn"
#Allow LAN traffic to connect to FreeNAS - change xxx.xxx.xxx.0 to match your network, ie 192.168.0.0 or 10.0.0.0
pass in on $ext_if from xxx.xxx.xxx.0/24 to any keep state
#Allow traffic out from the LAN
pass out on $ext_if from any to any keep state

Enable the Firewall

4. Edit /etc/rc.conf and add the following.

vi /etc/rc.conf

pf_enable="YES"
pf_rules="/mnt/Files/hacks/pf.conf"
gateway_enable="YES"

5. Start up the firewall and see if it works

service pf start

Your should get this as your output:

Enabling pf
No ALTQ support in kernel
ALTQ related functions disabled

Now check to make sure it is working:

service pf status

Your output should be something like this:

No ALTQ support in kernel
ALTQ related functions disabled
Status: Enabled for 0 days 00:04:55           Debug: Urgent

State Table                          Total             Rate
current entries                       29
searches                         1040038         3525.6/s
inserts                               95            0.3/s
removals                             109            0.4/s
Counters
match                               1093            3.7/s
bad-offset                             0            0.0/s
fragment                               0            0.0/s
short                                  0            0.0/s
normalize                              0            0.0/s
memory                                 0            0.0/s
bad-timestamp                          0            0.0/s
congestion                             0            0.0/s
ip-option                             22            0.1/s
proto-cksum                            0            0.0/s
state-mismatch                         0            0.0/s
state-insert                           0            0.0/s
state-limit                            0            0.0/s
src-limit                              0            0.0/s
synproxy                               0            0.0/s

6. Start IP Forwarding without restarting your server

sysctl net.inet.ip.forwarding=1

Then make sure to go into the FreeNAS gui, click on System, Click on Sysctls.  Then add that same value so it survives reboots.

Screen Shot 2014-05-14 at 10.43.22 AM

Make your changes stick

7.  Add your rc.conf changes to /conf/base/etc/rc.conf

vi /conf/base/etc/rc.conf

pf_enable="YES"
pf_rules="/mnt/Files/hacks/pf.conf"
gateway_enable="YES"

To really make these changes stick, follow my guide on the subject.

Clean Up

8. Make the filesystem read only again

mount -ur /

Final Thoughts

An added benefit of setting up a firewall this way is that it will let you route to other computers on your LAN over your VPN.  Hope you all enjoy this and let me know how things work out for you.

FreeNas – recovering from an upgrade that wipes out OpenVPN


Upgraded a couple of my FreeNas servers from 8.3.0 to 8.3.0 p1 last night.  That part went great.  What didn’t work as well is when I got to work today and tried to connect to the VPN on one of them.  Seems that the bone-head in me forgot that upgrading the FreeNas OS would wipe out the modifications to rc.conf to start OpenVPN as well as wipe out the entire /usr/local/etc/openvpn directory with all of the keys and openvpn configuration in it.

The nice folks over at FreeNas.org have anticipated me being a doof and created the upgrade process such that the old OS is preserved.  You can easily roll back to that if you are infront of your server by rebooting and hitting F2 (possibly F1) at the boot loader.

My problem was I wasn’t infront of my server and needed the VPN up ASAP.  So, I reached out to my big brother (Josh Paetzel) and magically the answer appeared in my inbox.

  1. ssh into the server.
  2. su to root
  3. mount
  4. You are looking for this “/dev/ufs/FreeNASs2a on / (ufs, local, read-only)”If it says that, then your old install is /dev/ufs/FreeNASs1a, if it says FreeNASs1a then the old install is FreeNASs2a
  5. Either way, mkdir /mnt/oldinstall
  6. mount /dev/ufs/FreeNASs1a /mnt/oldinstall 
  7. or mount /dev/ufs/FreeNASs2a /mnt/oldinstall
  8. ls /mnt/oldinstall/conf/base/etc/local/openvpn
  9. mount -uw /
  10. cp -r /mnt/oldinstall/conf/base/etc/local/openvpn /conf/base/etc/local/.
  11. grep openvpn /mnt/oldinstall/conf/base/etc/rc.conf >> /conf/base/etc/rc.conf
  12. Either reboot or repeat steps 10 and 11 replacing /conf/base/… with /etc/rc.conf and /usr/local/etc/openvpn

A great big thanks to Brother Josh on this one.  Always there saving me from myself.  Hope this helps someone else in some way, shape or form.

If I build it


I have the opportunity to build a server for a company that I’ve been doing some freelance work for.  It will mostly be a file server.  Originally it was just going to be an easily expandable archive/backup of digital images. Lots and lots and lots of digital images. (3-4 TB, I know…to some of you that isn’t very much) It was really only going to be accessed by a single employee on a daily basis.

Given that brief description of desires, I thought a FreeNas box would be perfect. Set up a VPN, a ZFS filesystem and some shares and be done.

Now, things have changed a little. The thought has been brought up that maybe the fileserver should become the primary repository for images. Currently all images are uploaded to the “cloud” by the various photographers and then downloaded to a disk as a backup. The “cloud” provider has been having some performance issues lately and has negatively impacted the back of house production workflow. Thus the idea to switch things up.

So the question is, does it still make sense to use FreeNas? The main issue as I see it is there is a desire for a super simple user-friendly way for all the photographers to be able to upload images to the fileserver. I thought that SFTP would be a winner, but sadly, some of the locations that the photographers shoot at have restricted web access. Also, the photographers have had problems in the past with FTP.(operator error mostly, but still a real issue) That makes me think that WebDAV might be a good solution, as it runs over port 80. However, I’ve had problems with it when trying to use certain OS’es to access a webdav share. For example, the apache webdav implementation under FreeBSD 8.x does not play nice with Mac OS X 10.6.x.

Another thought is to have all the photographers connect to the VPN and then do standard file sharing. That will mean running the VPN over a non-standard port to get around the restrictions at some of the sites. Not a deal breaker. However, I am not looking forward to trying to train all the photographers on how to use the VPN.

Last thing that I’ve looked at is a solution like ajaXplorer. I have no experience with this type of web software other than playing with it for an hour in a FreeNAS VM. It might be the best solution…just don’t know enough about it to make an educated decision.

So, if anyone is out there and listening, I could use some advice.