Tuesday, September 15, 2020

FreeNAS What is a L2ARC Cache and why is it useful?

In FreeNAS nothing beats RAM. A general rule of thumb is a minimum is 1GB of Ram per TB of storage but might need to be adjusted depending on workload/application.  Personally I prefer to have 2 GB RAM to 1 TB of disk if possible. Generally in the FreeNAS (truenas) community it is well known that it is recommended that FreeNAS run on 32GB of ECC RAM minimum, however I have found that if you are using a the 2:1 Ram ratio you can use non-ecc ram and FreeNAS runs fine though is not recommend by FreeNAS; but you work with what you got.

FreeNAS uses ZFS which provides a read cache in RAM, known as the ARC, to reduce read latency. FreeNAS adds ARC stats to top(1) and includes the arc_summary.py and arcstat.py tools for monitoring the efficiency of the ARC. 

If you use a L2ARC cache it is recommended using an SSD as a cache device.  ZFS uses it to store more reads which can increase random read performance.  If you are using a low ram system you shouldn't use a L2ARC, it will not increase performance, and in most cases you will actually hurt performance and could cause system instability.  If the hit rate of the ARC is below 90%, the system can benefit from L2ARC. If the ARC is smaller than RAM or if the hit rate is 99.X%, adding L2ARC to the system will not improve performance, the L2ARC is really only useful if your asking for the same data over and over again. 

As far as selecting appropriate devices for L2ARC, they should be biased towards random read performance. The data on the L2ARC is not persistent, and the L2ARC will be cleared on a reboot.  If you have a drive failure on your L2ARC device there is no need to worry ZFS is resistant and only uses it for read caching; your NAS will continue to function, however read performance will most likely be affected.  There is no need to provision, mirror or otherwise make L2ARC devices redundant, nor is there a need for power protection on these devices.

For example on some of my backup NAS systems I've build where I am moving over 500+ GB of data within 20 minutes my L2ARC cache gets hit quite a bit.  However it depends on what your doing.  It might be more beneficial to have a mirrored SSD pool for more active data and spinning drives for less active content then having a L2ARC Cache if the data needs to be read.

I would suggest using the tools in FreeNAS and monitoring your L2ARC's effectiveness using tools such as arcstat. If you need to increase the size of an existing L2ARC, you can stripe another cache device using Volume Manager. The GUI will always stripe L2ARC, not mirror it. As stated above the L2ARC data is not persistent.

I usually add a L2ARC if I have a spare SSD handy; with a 2:1 RAM ratio my experience is that it does help more then hurt but it depends on what your setup is and what your using it for.  Most of the NAS Systems I use are used for smb backups, and I'm backing up severs virtual disks which are always changing and moving them from NAS to NAS.  Having an L2ARC cache doesn't help there but with a file server that servers out files that don't change much the L2ARC does help with the read performance.

Note: that a dedicated L2ARC device can not be shared between ZFS pools.

Setting up Microsoft for Non-Profits

Google has been given access to GSUITE for a while now and I've setup several non-profits with this and do occasionally help them administrate it if required.  Now Microsoft is getting in the game (and this is a good thing) now giving Office 365 for free for non-profits.

Microsoft 365 is now free for non-profits
This post will go though what is required to apply and (hopefully) get approved for Microsoft 365 for Non-Profits.  I highly recommend before going forward you go thought the https://www.microsoft.com/en-us/nonprofits/ site and verify your eligibility.  The plans that are available for free are the Office 365 Nonprofit Business Essentials and 

Before you setup your Microsoft 365 account I recommend you have the following:
  • a techsoup account
  • a domain, ie. $yourdomain.ca
  • an email on that domain that can receive email.  This may require setting up an email alias if you don't have your own server
  • a way to verify your registration.  I recommend using a business or corporation number
  • picture of registration information
When you have all this information setup then you can go ahead and signup for Microsoft 365 Business for Non Profits.  When signing up I used a image of the registration information.  In the case of the setup I did I used the corporation number, and your techsoup validation token.

For more information about what is included in the free version of office 365 for non-profits.  https://www.techsoup.org/support/articles-and-how-tos/what-you-need-to-know-about-microsoft-office-365-nonprofit

Once you've filled out all the information you will get a verification email stating that the office 365 application is in progress.

Office 365 Verification
Once your organization has been verified you will get an email informing you to some of the services that are available for your organization.

When you go to login you can use the non-profit portal to login.  Typically you will be logging in with the format $user@$domain.onmicrosoft.com

Once you login to the non-profit portal you see your status and quick links for github, linkedin, and more Microsoft services that are available to you.

When you sign in to office.com you will be taken to a dashboard.

From here you can go into the office admin, non-profit portal, etc.  At least for this non-profit I am showing you the dashboard for they are allowed up to 25 unique users.

My experience with google there is no limit on the number of users you can have, but if you want Microsoft's products and services this is a fantastic service which I am sure will cut into google apps for non-profits adopters.  I found the setup and verification quick and fairly easy.  You are required to setup 2FA for the administrator account and for that I use and recommend Microsoft Authenticator available on Google Play and iOS

Friday, September 11, 2020

Installing a small footprint version of Ubuntu Desktop for Ubuntu Server

When I install a Linux server like Ubuntu Server, I occasionally do require a desktop but I don't want all the additional software that comes with the gui like LibreOffice, Thunderbird, etc.  Don't get me wrong, this is all great software for a client desktop but it isn't something that one might want on a server.  To achieve a small desktop footprint I do the following:

  • Get and install updates
    "sudo apt-get install update"
  • Get and install upgrades
    "sudo apt-get install upgrades"
  • Install lightdm
    "sudo apt-get install lightdm"
  • Install Unity
    "sudo apt-get install unity"
  • Install ubuntu desktop

    *** Important update ****
    When I went to setup a new lite version of ubuntu server when I tried to use the latest version of Ubuntu Server 20.04.02 and ran 

    "sudo apt-get install --no-install-recommends ubuntu-desktop"

    I got the following error

    "Command line option --no-install-recommends is not understood in combination with other options"

    In previous versions of Ubuntu the command works fine such as in 16.04.07 but in Ubuntu 20.04.02 I had to use

    "sudo apt-get install ubuntu-desktop-minimal"

    So please note this important change if you are trying to create a minimal version of a linux desktop; this may also affect other distros.

    Then continue with the other installs

    "sudo apt-get install compizconfig-settings-manager"
    "sudo apt-get install firefox"
    "sudo apt-get install net-tools"

If you want to pin the terminal application to the launcher, but the launcher won't show up in your applications though the gui.

No Terminal Application showing up

To add the terminal to the launcher is to right click on the desktop and open the terminal

Once the terminal is open right click on the terminal in the launcher and lock to launcher

Friday, September 04, 2020

Setting up Papercut print release system for Linux

Ramsey Public Library Envisionware Print Release System
In December of 2019 an organization was expanding and opening a new location.  This post is part 1 of a 4 part series which will go though the investigationThey offer public printing services and were using a public printing solution from Envisionware called LPTONE.  I did a previous post on this software in 2018 with setting up a release station with the Envisionware LTPONE software.    

However after investigating upgrade options and the cost for adding an additional location and coinop for the release station it was not a cost effective solution since new hardware would be required for both locations and there was just not the budget for it.  Not to mention the difficulty of managing two release systems that were non-centralized.

Ramsey Public Library uses an envisionware print release system, I know from the experience I've had with it; it's ok but it is quite lacking and expanding they system it is currently on to windows server from windows 10 pro would be quite the expense as it would require more then 2 machines with server if we ever got over 30 clients and we were right on the cusp of being.  

The organization had service contracts with Toshiba for managing the maintenance and materials of their printers which also included a ITC 5400 coinop, which was attached to one of the copiers for paid copies; so I investigated if that company had any kind of print management software that could use the same coinop and if the coinop could do double duty for the coping and the printing.  After contacting ITC and verifying that yes indeed the 5400 coinop can do double duty for the printing and coping payment; however they use a 3rd party software for the print management called papercut.

Now Papercut is a centralized print management solution, that works on Linux or Windows and is actually pretty light on system requirements.  At a glance you would think for just managing client side printing the NG version would all that is required however it didn't quite do all that was desired from the printing side so it was decided to go with the MF version.  The project started with a decision to go with a Ubuntu 18 LTS virtual server on Hyper-V to host the papercut MF software.

You can view Papercut's System Requirements I found on Hyper-V these settings work pretty good for a workgroup of of about 40 clients.

Processor: 2 Core

Minimum 4GB of Ram (max of 16GB) Dynamically Assigned

Disk Space: 
3GB for Log, 100GB free disk space, between 60 and 500GB recommended.

Here are the CDN links for the papercut software if you want to try it yourself.



Setting up Papercut on Ubuntu Server 18.03 LTS / 20.04 LTS

There are a couple things you need to know before you can install papercut on Linux.

  1. The "ROOT"/Administrator must be called papercut
  2. The user must have sudo access for the install
So when you install your username must be papercut, otherwise you will have serious issues with your install, with permissions etc.  

For this install I'm going to use the following credentials:

  • username: papercut
  • password: papercut
and I am installing open ssh and powershell.

Once we have finished our base install we will need to install the following (for a slimmed  down version):
  • Get and install updates
    "sudo apt-get install update"
  • Get and install upgrades
    "sudo apt-get install upgrades"
  • Install lightdm
    "sudo apt-get install lightdm"
  • Install Unity
    "sudo apt-get install unity"
  • Install ubuntu desktop
    "sudo apt-get install --no-install-recommends ubuntu-desktop"
    "sudo apt-get install compizconfig-settings-manager"
    "sudo apt-get install firefox"
    "sudo apt-get install net-tools"
If you don't want a slimmed down version just use "sudo apt-get install ubuntu-desktop"
  • restart
  • Install and configure cups
    "sudo apt-get install cups (should be already installed)"
  • add the papercut user to the printer admin for cups
    sudo usermod -a -G lpadmin "$USERNAME"
  • Install Samba
    "sudo apt-get install samba"
  • add the papercut user to the sudoers list
    sudo vi /etc/sudoers
    papercut ALL=(ALL:ALL) ALL
  • Install papercut
    chmod 777 "PATH TO PAPERCUT".sh
You can watch a video on the full process on my youtube channel here. https://youtu.be/9re8L6uWc94

How to fix CURL call imporitng an RSS feed on a site blocking CURL calls

There is a 3rd party service provider that my organization uses called bibliocommons.  They have these nice book carousels.  However the car...