How to fix CURL call imporitng an RSS feed on a site blocking CURL calls

There is a 3rd party service provider that my organization uses called bibliocommons.  They have these nice book carousels.  However the carousels are not very customizable and are only available for specific lists made by Bibliocommons. 

So I wrote a php rss reader that takes an RSS feed that is produced by the page list and built a carousel that is more customizable and updates it's self once a day (see below)

The carousel I wrote has all the art at the same height; and it puts the name of the book under the art which makes it more accessible and still links to the book in the catalogue just like the other carousel.  It has been working fine for 6 years but not after a major DDoS attack on the vendor; they are using cloud front to stop scripts from hitting the site using CURL and broke my carousel (see below).

Even after asking the vendor to whitelist the webserver things were not progressing on getting the issue resolved so; comes sublime text edit (my all time favorite coding tool).  So I thought I would see what exactly is going on since I could access the RSS feed just fine in my web browser but my script was receiving 301 - 307 errors (whatever the server felt like throwing).

So troubleshooting this I found two issues.  One was a full url that was in the node value and not being escaped and a USER_AGENT detection which was causing the script from accessing the RSS feed; which seems silly because I would think you would want users to access RSS feeds.  Since my browser was able to access the RSS feed; I determined that they must be doing some sort of detecting; which they were.

CODE

<?PHP  

$ch = curl_init();

  $url = "$RSSURL";

  curl_setopt($ch,CURLOPT_URL,$url);

  curl_setopt($ch,CURLOPT_USERAGENT,'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.79 Safari/537.36');

  curl_setopt($ch,CURLOPT_HEADER, 0);

  curl_setopt($ch,CURLOPT_NOBODY, 0);

  curl_setopt($ch,CURLOPT_RETURNTRANSFER, true);

  curl_setopt($ch,CURLOPT_FOLLOWLOCATION, 1);

  curl_setopt($ch,CURLOPT_CONNECTTIMEOUT ,5);

  curl_setopt($ch,CURLOPT_TIMEOUT, 20);

  $response = curl_exec($ch);

  curl_close ($ch);

file_put_contents("$XMLDATAFILE.xml", $response);

?>

So the key was adding the CURLOPT_USERAGENT to get access to the RSS Feed; I randomly chose the user agent but you could randomize it in a variable so you wouldn't have the same useragent hitting the server every time if you wanted something more random.  

You can get some sample agents from deviceatlas.com,  Once added to my CURLOPT; my carousel started to work again and the XML errors where corrected using the file_put_contents command into a different xml file and reading that XML file.

How to migrate PFSense Over to KEA DHCP Server from ISC DHCP Server

I am a PFSENSE User and I manage PFSENSE for some other organizations and the time has come to make the switch for the DHCP Server over to KEA from ISC.

Why switch to KEA from ISC?

• ISC will no longer be supported
• KEA has some nice High availability features for IPV4 & IPV6
• Robust Host Reservations which are per subnet vs global, but global reservations are supported.

ISC has some documentation and tools for migrating over to KEA from ISC.  PFSense has a warning on the firewalls which can be ignored at your own peril.

Now in PFSense if you try to just "switch" over you may encounter an error.  You can see that my ISC Server is running fine but when I switch over to KEA DHCP "Breaks"

The issue is caused by the NTP Server settings in the DHCP Server, not sure why they are there, perhaps if you wanted to setup your own linux NTP server or something for time syncing or an AD Server then you could just put in the IP of the machine.  Still with how you now setup NTP in PFSense it seems a little redundant to have that there.  At any rate you need to either remove the NTP DNS server settings from the DHCP server settings unless they are an IP address.  So for myself I setup the NTP server to the localhost 127.0.0.1 or the main ip or you can do both.  

With that setup we can modify our NTP server in PFSense located under services

It is recommended setting up 3 to 5 ntp servers to sync with.  For myself I decided to go with

1. time.apple.com
2. time.windows.com
3. time.google.com
4. time.cloudflare.com
5. time.nist.gov

I have set time.apple.com and time.windows.com as my preferred NTP Servers

With that setup and done, I ssh'd into my firewall to check the time.  

I ran the command date +"%T" 

and when you check the general setting for the time servers you see the NTP Servers we setup in our NTP Service

With these settings your KES migration should be complete and the DHCP Server should work flawlessly.  I had no issues with the migration once I removed my NTP servers using DNS.  Time has been in sync with no issues.

How to disable MDM/DEP notifications from macOS Big Sur to Sonoma

This post will allow you to disable/prevent MDM/DEP Notifications and macOS POPUPS forcing you to join an organization's MDM.  This would be handy if a computer was purchased and not removed from Apple's MDM.

Restart the mac in Recovery Mode Restart your Mac then hold down the Command + R keys together until you're in the Recovery Mode

From the menu bar Click on Utilities then select: Startup Security Utility

A popup appears with 3 choices; select (No security) and restart
NOTE:  There is no confirmation notification or button to press

When the system reboots hold down Command + R to get back into Recovery Mode.

Click on Utilities from the top menu bar then select Terminal

In terminal type: mount then press enter/return

A list of devices will show up in a list.  You will want to use the volume that specifically says "/Volumes/Macintosh HD"

In my case it is /Volumes/Macintosh HD (/dev/disk3s4) 

NOTE: it's not "/" (root), and it's NOT /Volumes/Macintosh HD - Data

In Terminal Write:

1. umount /Volumes/Macintosh\ HD
2. mkdir /Volumes/Macintosh\ HD
3. mount -t apfs -rw /dev/disk3s4 /Volumes/Macintosh\ HD
4. cd /Volumes/Macintosh\ HD/System/Library/LaunchAgents
5. mkdir xtemp
6. mv com.apple.ManagedClientAgent.* xtemp/
7. mv com.apple.mdmclient.* xtemp/
8. cd ../LaunchDaemons
9. mkdir xtemp
10. mv com.apple.ManagedClient.* xtemp/
11. mv com.apple.mdmclient.* xtemp/
12. csrutil authenticated-root disable (this will Turn off Signed System Volume SSV)
13. bless --folder /Volumes/Macintosh\ HD/System/Library/CoreServices --bootefi --create-snapshot
 (this will Save the current disk status in the boot snapshot)

Now you can restart your Mac, and DEP notifications are disabled.

Hyper-V Cluster Node Keeps Randomly Going Down

Over the last few weeks I had an issue with a Hyper-V Cluster node randomly going down causing issues with my Hyper-V cluster.

Looking at the event logs it looks like it is an issue with the network driver.  It was giving a number of errors including mac address duplication and non-operational status in the logs as shown below.

A few different things were done to try resolve the issue, such as a driver re-install, driver update, and a SFC scan.  However the problem persisted for a randomly continuing over a day or two coming on and offline normally for a few hours then dropping and reconnecting.  What I had to do to resolve the issue was in the cluster manager put the node offline then uninstall the network driver; do an offline disk check then reinstall the network driver.  Once that was done and the cluster node came back up and has been stable ever since.

How to setup a Rustdesk server

To see a video of the Rustdesk installation visit https://www.youtube.com/watch?v=t7UobpjDsRY and if you like the content please like and subscribe.

What is rustdesk? Rustdesk is an open source remote access and remote control computer software, allowing maintenance of computers and other devices.  For years I have used the free version of teamviewer to help family/friends with various computer issues but in the last few years teamviewer has cut the number of sessions you can have along with how long you can have those sessions for.  I have used the enterprise version of teamviwer and it is full of great features which I would say if you can afford to get teamviewer you should it will make your life a lot easier.  If you can't than rustdesk is a really good alternative.

For a while I was using the free server provided by rustdesk but a few events reciently like a host suspension and a major ddos attack against their server have encouraged me to spin up my own rust desk server.

I have done this a few times now; and there are things in the server setup process where if you deal with them before hand it will make your setup go really well and quite quickly.  So the when deploying a rustdesk server this is how I now do the setup and deployment.

Questions that must be answered!

1. Where is the server going to live, what is the IP address and what is the server's name?
   
   This seems self-explanatory but is really important; especially if you don't want to be messing with host files and other things in Linux.  Decide where the machine will live on what network and what the name will be.
   
   So for example, the machine is going to live on a SMB network where the ip address is 192.168.2.0/24, with a reserved IP of 192.168.2.137 and a name of rustdesk.smb.ca (smb is the company's registered domain)
   
   
2. What is the hypervisor?
   
   For this setup I will be using Microsoft Hyper-V as our hypervisor platform of choice.
   
   
3. VM OS?
   
   For this setup I will be using Ubuntu Server 22.04 LTS

Steps for installation.

First setup your VM and set the network adapter to a static address.

Get the static address and assign that to your router.  If you are wanting to give it a reserved address then go ahead and do the install but be sure to setup the DNS to what you want it to be; as it is difficult to change it later.

Once done start the install of the system.  For the install you will want to do a minimum system install and you will want to also allow for ssh access along with the install of the live patch system and powershell

Be sure to make the server name what you want it to be on your network.  It is difficult to change the name after the fact.

Once installed run your updates and I like to install the linux firewall ufw and vim.

Now we can setup the server and install rustdesk.  There is a good guide on https://github.com/techahold/rustdeskinstall  which is more/less the guide we are going to use to install the server.

Steps:

1. Configure the firewall
   
   ufw allow proto tcp from YOURIP to any port 22
   ufw allow 21115:21119/tcp
   ufw allow 8000/tcp
   ufw allow 21116/udp
   sudo ufw enable
   
   
2. download and run wget https://raw.githubusercontent.com/dinger1986/rustdeskinstall/master/install.sh
   chmod +x install.sh
   ./install.sh
   
   
   You will want to setup the server with DNS for the first option and also include the http server for the second option when you run though the installer.  For the DNS you should keep with the fomat "Servername.domain" so if my machine is called rustdesk it will be rustdesk.domain
   
   ****IMPORTANT****
   
   The installer will give you a summary which will include your login for the webserver and the public key for the rustdesk client so users can connect to your rustdesk server.  Be sure to either write down this important information or take a screenshot.
   
   
3. After the install has finished you should reboot the vm and run the update script for rust desk

After the reboot the site should be accessible on your lan via DNS and/or IP at port 8000 so if my DNS was setup as rustdesk.smb.ca; the webhost will be access at http://192.168.2.137:8000 or http://rustdesk.domain:8000

Now update your firewall to allow access to the rustdesk ports and so you can use your rustdesk from anywhere on the internet.

I have a WAN rule with the following (the rustdesk is an ip alias)

and I have a floating rule with the same information

Now we can add our server settings to the rustdesk client.

To do that you need to open rustdesk go to -> Setting -> Network and unlock the settings if required.

After you input your server settings if everything has been setup properly your rust desk will show as ready

Now you will only be able to connect to other users who are connected to another server or the public one provided by rustdesk; you will not be able to assist them.

References:

https://rustdesk.com/docs/en/self-host/rustdesk-server-oss/install/#set-up-your-own-server-instance-manually

https://rustdesk.com/docs/en/self-host/

https://github.com/rustdesk/rustdesk-server-demo

https://github.com/rustdesk/rustdesk-server

https://www.reddit.com/r/rustdesk/comments/17yevvf/rustdesk_self_hosted_setup_guide/

https://rustdesk.com/docs/en/dev/build/web/

https://github.com/techahold/rustdeskinstall

Setting up and Customizing a Windows 11 Start Menu without Microsoft Intune

This post will give you everything you need to customize the windows 11 start menu.  Now typically this is done with intune and Microsoft decided to significantly change the way it handles the start menu and changed it from using an xml file to using a JSON file which is to be used with intune.  Now if you don't use intune; you have a problem.  However this post will go though the options and making a start menu for your Windows 11 clients.  A plus for this is that you don't have to make changes or worry about how it will affect your windows 10 clients.

If your looking at wanting to make a custom Windows 10 start menu you can find that info here https://optionkey.blogspot.com/2018/02/upgrading-active-directory-for-windows.html

In Windows 11 to change to position of the start menu you can do with with a registry entry.

Add a registry Entry called "TaskbarAl" the key goes here 

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Advanced

Value name = "TaskbarAl" the l is a L

It is a DWORD 32bit setting, where 0 is aligned to the left and 1 is centered.

Now for customizing the start menu you setup it up like you would for Windows 10.  Customize it up as a single user then run the powershell command.  Now this has been designed to work specifically with intune and making it work without intune takes some work but here is how you do it.

Export-StartLayout -Path "C:\Layouts\Win11Layout.JSON"

 The result is the following JSON:

{"pinnedList":[{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\Google Chrome.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\Microsoft Edge.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\Firefox.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\Horizon 7.5.6.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\Adobe Acrobat.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\Word.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\Excel.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\PowerPoint.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\Publisher.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\OneNote 2016.lnk"},{"packagedAppId":"Microsoft.WindowsCalculator_8wekyb3d8bbwe!App"},{"desktopAppLink":"%APPDATA%\\Microsoft\\Windows\\Start Menu\\Programs\\File Explorer.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\VideoLAN\\VLC media player.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\paint.net.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\Notepad++.lnk"}]}

Open windows explorer and paste the following URL in the explorer

%LocalAppdata%\Packages\Microsoft.Windows.StartMenuExperienceHost_cw5n1h2txyewy\LocalState

It will take you to a folder and copy the start2.bin to a location where you can apply it with your ad controller; I put it in the NETLOGON folder on my AD Controller, if you need it to be local put it in a folder that is local.

Now we are going to have AD add a registry entry on our clients and there are two of them, we are going to add them to the User Configuration section of group policy.

The first one we are going to make is called "ConfigureStartPins_ProviderSet"

HKEY_LOCALMACHINE\SOFTWARE\Microsoft\PolicyManager\current\device\Start

It is a DWORD 32 bit value and the value should be set to 1.

Now we need to make a new Registry entry called "ConfigureStartPins"

Here is where we will be adding the JSON we got from the start menu.

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\PolicyManager\current\device\Start

In the value data we paste the JSON we exported out earlier.

{"pinnedList":[{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\Google Chrome.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\Microsoft Edge.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\Firefox.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\Horizon 7.5.6.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\Adobe Acrobat.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\Word.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\Excel.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\PowerPoint.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\Publisher.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\OneNote 2016.lnk"},{"packagedAppId":"Microsoft.WindowsCalculator_8wekyb3d8bbwe!App"},{"desktopAppLink":"%APPDATA%\\Microsoft\\Windows\\Start Menu\\Programs\\File Explorer.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\VideoLAN\\VLC media player.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\paint.net.lnk"},{"desktopAppLink":"%ALLUSERSPROFILE%\\Microsoft\\Windows\\Start Menu\\Programs\\Notepad++.lnk"}]}

Next we need to make a file to copy the start2.bat to the user profile when you login; I called the file startmenu.cmd and I am running it from the same location from the start2.bat which is in the NETLOGON folder, and we are going to put in the following code which we will run at logon.

copy "\\$DOMAIN\netlogon\start2.bin" "%LocalAppdata%\Packages\Microsoft.Windows.StartMenuExperienceHost_cw5n1h2txyewy\LocalState" /Y

This will replace the default start2.bat file with the configured one.

In group policy I am putting the following in the Scripts section of User Configuration

\\$domain\SysVol\$domain\folder\startmenu.cmd

Now on login your windows 11 clients will have a custom start menu.

References:

https://learn.microsoft.com/en-us/windows/configuration/customize-and-export-start-layout

https://www.ivobeerens.nl/2022/12/13/customize-the-windows-11-start-menu/

https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.management/get-itemproperty?view=powershell-7.4

https://www.thewindowsclub.com/move-windows-11-start-menu-to-left-side

TrueNAS Server Setup

I setup a new TrueNAS system to work with my lab server for storage.  The system is running on an older platform an Intel Haswell 1150.  The system has 32 GB of ram and as you can see below the system only has 6 sata ports.  I put in a Matrox 10Gig card for connecting my server 2022 system to the iscsi target so that left me with one PCI Express port to add a raid card or a card for additional sata ports. 

Now I am a big VANTEC fan, I have purchased a lot of enclosures and adapters from them over my IT career and they have worked really well and I came across this nice little card.

https://www.vantecusa.com/products_detail.php?p_id=286  it was at my local computer retailer memory express https://www.memoryexpress.com/Products/MX00120961 where it is retailing for $59.99.

This card requires at least a Gen3 4x PCI-E port, I know my board supported Gen3 PCI-E so I thought I would give it a try.

Initially I had issues getting TrueNAS to boot with the system encountering re-scan errors when trying to initialize the card.

(noperiph:ahcich3:0:-1:ffffffff): rescan already queued

After some troubleshooting; I determined that it had to do with the system cold booting because after a couple of reboots the system would boot and recognized all the drives.  I also noted that the card did not initialize until just before they system was about to boot.  The system registered all my drives, I have 3 SSD drives on the motherboard SATA ports and the rest of my 3+ TB drives running off the card.

So for the disk setup I have a 120GB SSD boot drive, 40Gig SSD Z-Log Drive, and 120GB SSD Cache Drive.  The rest of the drives are NAS drives mostly WD REDS and two 3TB seagate iron wolfs.  

I setup a SMB share to test how the system might perform

Though a 1Gig network the samba share registered the following through put

When I setup the iSCSI target on my lab server I will update this but for now I am pretty happy about this and anyone looking for an inexpensive controller for TrueNAS, I would  defiantly recommend this controller if your on a budget.  The throughput on the NAS is pretty much max out on my 1Gig network, and the NAS did not sweat a bit!