Setup VLAN with TP-LINK WR1043ND v1.6 (LibreCMC)

As mentioned before, I have flashed LibreCMC on TP-LINK WR1043ND v1.6 with WiFi disabled. It is my backup router.  Because my ISP support 2 IPs, and the new cable modem does come with 2 ethernet ports, I have one port connecting to my DIR-882 (using Padavan firmware) serving the main floor and my bedroom in 2nd floor with WiFi enabled.  The 2nd port is now connected to TP-Link WR1043 in the study room.  Now, I want to create a separate VLAN for miner, so, it is separated from my test equipment.  So, I update LibreCMC to 1.5.14.  It requires the "Keep Settings" to be unchecked for the upgrade.  Please check the link below before proceed.  Since it was a backup router, it does not matter for me if the settings are wiped.  

Releases - Gogs (librecmc.org)

There is not many guides in the internet for LibreCMC.  Since it is based on OpenWrt 19.x, I check the youtube guide below.  

How to Create a VLAN - A Beginner's Guide // OpenWrt Router (Up to 19.x) - YouTube

How to configure OpenWrt as Firewall for your home network and Guest Wifi and IPTables explained - YouTube

First step is to go to Network > Switches.  You will see VLAN 1 and 2 populated already.  Now, add a new switches with VLAN 5.  I pick LAN port 3 for VLAN 5 which will be used by the miner.  VLAN 1 will be used for test equipment (LAN port 1, 2 4). 

Second step is to go to Network > Interfaces.  Click "Add new Interface" to add a new interface named  VLAN_5.  Below are the rest of settings.

Protocol: static addresses

For my setup, "Create a bridge over Multiple Interfaces" is unchecked since WiFi is disable.  

Cover the following interfaces: select eth0.5 (automatically created when VLAN 5 is created in previous step)

Then, go to Network > Interfaces again.  Click on the VLAN_5 interface and apply the settings below.

Protocol: Static address

Bring up on Boot: checked

IPv4 address: default gateway for subnet VLAN 5

IPv4 subnet mask: select the subnet mask

Then scroll down to DHCP server section.

Select the range for DHCP client IP address and the lease time. 

Click on Advance Settings under DHCP section.  Make sure Dynamic DHCP is checked.  

Now, click on Physical Settings tab.  In my case, Bridge Interface is unchecked since WiFi is turned off and the correct Interface eth0.5 is selected for VLAN 5.  

Next, go to Firewall Settings Tab.  Create a new firewall zone called VLAN5 for VLAN 5.  

Lastly is to config firewall.  For VLAN_5, I don't want the miner to reach the router other than getting IP and DNS resolution but will have access to internet.  Also, it cannot reach to VLAN 1.  

So, setup VLAN5 zone to allow VLAN5 forwarding to WAN to get internet access.  Input and forward are set to reject and Output is set to accept.  

To allow client (miner) in VLAN_5 to reach to router for DHCP and DNS, setup a rule to "Accept Input" from VLAN_5 to router IP at 53, 67 and 68.

There are other exception rules predefined.  You can disable them based on your requirement.  

Setup Tumbleweed as physical host for VirtualBox

I decide to try Tumbleweed on the same old FX-8320 host.  Installation is similar to LEAP 15.4.  For GUI option, I choose Generic Desktop instead of XFCE Desktop.  XFCE is running quite smoothly in LEAP 15.4.  I decide to try Generic Desktop this time since it is only a physical host.  

Follow the same link below, I try to install driver version G05 for my old GT630 (does not work with LEAP 15.4).  

SDB:NVIDIA drivers - openSUSE Wiki 

Reboot the host and it suggests to install the previous version G04.  Follow the wiki link above and install G04.   

 

Below are my software pick for the host.  

File Manager: PCmanFM

Terminal: termit

Text Editor: Leafpad

Graphics: Pinta (take and edit basic screenshot)

GRsync: sync my USB drive for backup.  

USB wireless 8812bu adapter is discovered correctly.  No driver installation is required.  

Only issue is NetworkManager icon is not available for Generic Desktop.  First I install wicked and other tools.  It works fine but I decide to try to get NetworkManager working.  Then I remove wicked and not sure why, there is no networking at all.  So, I rollback the system to the point before wicked is installed.  

Once NetworkManager-Connection-Editor is installed, I have the simple GUI to manage my network connection.  If your connection does not change much, wicked is just fine. 

Then install VirtualBox.  Reference link is How to install Virtualbox on OpenSUSE Tumbleweed & Leap - Linux Shout (how2shout.com.

sudo zypper install virtualbox

I find out 7.0.10 is installed.  

Once completed, add the user account to group vboxusers

sudo gpasswd -a userID vboxusers

Then, logoff and log back in.

Open VirtualBox.  If you are asked to enable USB passthrough, just click Enable.  

Download the VirtualBox extension pack from VirtualBox site.  Same as before, it does not work with GUI installation.  I install it thru CLI (see command below)

VBoxManage extpack install extpack_7.0.10

I decide to go with KVM/QEMU this time.  Will create a separate post for that. 

Isilon multiscan moves data between tiers unexpectedly

 After Isilon was implemented 2.5 yrs ago, we decided to stop SmartPool jobs and use combination of IndexUpdate and FilePolicy job to save resources on the Isilon.  We have 4 SSD nodes handling front end traffic, 4 hybrid nodes handling replication plus some test traffic and 4 SATA nodes handling the backup.  Initially, SATA nodes were deployed with insufficient memory and caused couple of outages during NDMP backup.  After max the memory on SATA nodes, there are no more issue after that.  

Because we want to keep as much data as possible on SSD, we won't run FilePolicy until SSD used space is over 70%.  After firmware update (nodes reboot), we saw data move between Tiers with multiscan kicking in auto.  That's about 2 yrs ago.  Checked with support and they said it's ok to kill it.  So, we just killed the Mutliscan job.  Recently, we finally had our first disk failed on the production Isilon and Multiscan was kicked in auto.  Same as before, we saw data moving unexpectedly to SATA tier.  I suspected if that's related to FilePolicy.  However, I was told it should not by support.  After case with support for a long time, we finally got the suggestion from higher level of support to run SmartPool job periodically.  So, I play with the DR Isilon since it is not busy.  Using IndexUpdate and FilePolicy job to move the data and compare the tier usage multiple times, each tier finally reached to the utilization I want 60-70% for SSD and below 60% for SAS pool (the spillover pool).  Then, kick off MultiScan job.  Now, I don't see any more data move between tiers with MultiScan job.  

In the future, I will adjust the FilePolicy each month and then kick off the SmartPool job once a month just in case Multiscan is started due to failed HDD or node replacement.

===========================================================================

Update Sep 25, 2023

For Production Isilon, after IndexUpdate completes, I run FilePolicy then Multiscan job.  I still see data moves to SATA pool.  So, I run IndexUpdate -> FilePolicy -> SmartPool job.  Same things.  Looks like there is discrepancy between IndexUpdate + FilePolicy and SmartPool job.  

This time, I just run SmartPool job and kill it once SATA pool reaches 85%.  Then adjust the FilePolicy and rerun it SmartPool job.  After 3 tries, I finally see the results I want.  Why there is a discrepancy between FilePolicy and SmartPool job, I have no idea.  

From now on, I will adjust the FilePolicy once every 2 months and run the SmartPool job.  For DR Isilon, since it has far less data, I will adjust FilePolicy once every 3-4 month and then run SmartPool job. 

Cable Modem TC4400 overheated?

Switch to TC4400 cable modem for a few months because DOCSIS 3.0 will not be supported by my cable company.  This is the only one not using Puma chipset and supported by my cable company.  It worked fine until recently.  The internet connection drops few times day.  When I touch it, the modem is really hot.  The quick fix is to put a little 15mm by 15mm fan on the top of modem to draw the heat away from it.  Now, it is much better and has not experienced any more connection drop.  

How long to quick format a lun in Windows

 I always get that question for the SQL or file cluster RDM disks.  Windows admin and DBA thought the format hung but it indeed took long time to quick format a lun.  Last week, Windows admin complaint it took approx an hour to quick format a 7TB lun in EMC PMAX for a Windows 2016 / 2019 VM.  

I just completed a test this morning.  For a physical Windows 2016 server, it took about 10-15 min approx to quick format a Windows 1 TB lun in PMAX.  Do some research in the internet.  The result provided by partitionwizard.com suggests it indeed takes long time to format a big lun (see below also).  

• How Long Does It Take to Format a 1TB Hard Drive: Performing a Quick Format on a 1TB hard drive takes about 20 minutes. If you select the Full Format, it could take you up to 1 hour.
• How Long Does It Take to Format a 2TB Hard Drive: Again, we perform a Quick Format on a 2TB hard drive, it can be done in about 30 minutes. However, a Full Format can take up to 3 hours. If this hard drive stores a chunk of data, it could take you a half day.
• How Long Does It Take to Format a 4TB Hard Drive: To a certain degree, 4TB is a large hard disk that will take quite a long time to format. So, you’d better select a Quick Format if you want to save time. This is because fully formatting a 4TB hard drive can take you a whole day and even more

This give you an idea even quick format in Windows will still take some time.  

Our Wintel team will test the format time following EMC article 000062689 on the PMAX.

The cause is "trim and unmap" feature is on.  So, just temp turn it off before formatting new lun and enable it back once done.  

On the windows host, disable the SCSI TRIM and Unmap feature for the duration of the format. Use fsutil command from the command line

1) To verify the current setting, using a Windows CMD window on the Host, run:   

fsutil behavior query DisableDeleteNotify

DisableDeleteNotify=0 -indicates the 'Trim and Unmap' feature is on (enabled)

DisableDeleteNotify=1 -indicates the 'Trim and Unmap' feature is off (disabled)

2) To disable, issue the command:   

fsutil behavior set DisableDeleteNotify 1

3) Once formatting is complete, re-enable the feature using command:   

fsutil behavior set DisableDeleteNotify 0

It may impact Linux as well.  See thread mkfs is extremely slow. 

To run mkfs without trim, use the -K option on XFS and -E nodiscard on ext4

XFS

mkfs.xfs -K /dev/sdx 

EXT4

mkfs.ext4 -E nodiscard 

Warning: Only use -K or -E on new volumes with no existing data.

Using the -K or -E options on drives with existing data, will cause the space to be wasted until the data is overwritten.

  

Install Open VM Tools on Linux VM

Depending on the distro of Linux, I have tough time to install VM Tools on some of them.  The only other option is to install open vmtools.  

Follow VMware instructions to install the open vmtools.

If GUI is required on the VM and assume it is Debian distro, install open-vm-tools-desktop.  

sudo apt-get install open-vm-tools-desktop

Otherwise open-vm-tools will be sufficient.

sudo apt-get install open-vm-tools  

No Port Down email alert from DCNM

We used to have SNMP to monitor all FC port down event.  However, it causes a lot of un-necessary calls from NOC after hours.  Because we have most of the prod servers setup with 4 HBAs, we decide to use SMTP alert to email to storage team members only.  However, it never seems to work with DCNM.  We did see Port up alert but not for Port down regardless of what settings we choose.  

After opening a ticket with support, we were told the setting was disable by default.  If you want to receive email alerts for port down event, go to Server Properties of DCNM.  Look for event.linkDown.log and set it to True.  By default, it is set to False.  Then restart the DCNM services and port down alerts will be sent to your SMTP server depending on your setup.  

Cisco bug CSCvz61883

Check the bug info at Cisco site for CSCvz61883 / EMC article 000197332 and it applies to the MDS 9700 32G FC modules in our environment.  The DS-X9648-1536K9 linecard in MDS 9700 can be affected at almost exactly 468 days of uptime.  

Our MDS 9710 with 32G linecard are running 8.1(1a) with supervisor 3 and 8.4(1a) with supervisor 4.  Runs the command below against the 32G linecard in MDS9710 to see module uptime.  In our environment, they reach 426 and 417 days respectively.    

slot x show system uptime   

8.4(2d) does provide the fix to the issue but it is quite new.  Open a ticket with support and the recommendation is to upgrade to 8.4(2c).  This will reset the module uptime back to zero and buy another 400+ days.  With 8.4(2), there is a workaround to reset the uptime for the device non-disruptively.  You can consider another firmware update after 400 days on the FC switch.  

Finished the upgrade to 8.4(2c) last week.  Upgrade to 8.4(2c) on MDS 9710 with supervisor 3 is much faster than MDS 9710 with supervisor 4.  

=================================================================

Updated in late May 2023

Another reason we never considered 8.4(2d) was due to CSCwb29379 (see release notes 8.4(2e) below) 

Cisco MDS 9000 Series Release Notes, Release 8.4(2e)

8.4(2e) was available for quite some time now.  So, we upgraded to 8.4(2e) couple weeks ago to fix this problem permanently.  

Verifying that Every EMC Disk Arrays are Properly Attached to an EMC SMI-S Provider

Just update ViPR SRM to point to the new Windows 2016 with SMI-S provider.  However, SMI-S complains it cannot retrieve the info of the array.  Confirm ECOM is running and symcfg list shows the VMAX array.

Restart ECOM service and ViPR SRM can retrieve the stats from the array.   However, I try to find out how to confirm EMC SMI-S does detect the array. 

Follow the kb.
https://www.sentrysoftware.com/kb/KB1132.html

Below is the kb copied from the link. 

Objective

Monitoring EMC disk arrays requires to configure the EMC SMI-S Provider to make sure that the arrays are being discovered. EMC disk arrays are automatically discovered when they are locally attached to the EMC SMI-S Provider. This article explains how to make sure that the EMC SMI-S Provider discovers all disk arrays by using the TestSmiProvider tool.

Procedure

1. Logon to the server where the EMC SMI-S provider is installed
2. Run the TestSmiProvider.exe which can generally be found under “C:\ProgramFiles\EMC\ECIM\ECOM\bin”
   C:\Program Files\EMC\ECIM\ECOM\bin>TestSmiProvider.exe
   Connection Type (ssl,no_ssl) [no_ssl]:
   Host [localhost]:
   Port [5988]:
   Username [admin]:
   Password [#1Password]:
   Log output to console [y|n (default y)]:y
   Log output to file [y|n (default y)]:
   Logfile path [Testsmiprovider.log]:
   Connecting to localhost:5988
   Using user account 'admin' with password '#1Password'

   ########################################################################
   ##                                                                    ##
   ##                  EMC SMI Provider Tester                           ##
   ##   This program is intended for use by EMC Support personnel only.  ##
   ##   At any time and without warning this program may be revised      ##
   ##   without regard to backwards compatibility or be                  ##
   ##   removed entirely from the kit.                                   ##
   ########################################################################
     slp    - slp urls                     slpv    - slp attributes
     cn     - Connect                      dc      - Disconnect
     disco  - EMC Discover                 rc      - RepeatCount
     addsys - EMC AddSystem                remsys  - EMC RemoveSystem
     refsys - EMC RefreshSystem

     ec     - EnumerateClasses             ecn     - EnumerateClassNames
     ei     - EnumerateInstances           ein     - EnumerateInstanceNames
     ens    - EnumerateNamespaces          mine    - Mine classes

     a      - Associators                  an      - AssociatorNames
     r      - References                   rn      - ReferenceNames

     gi     - GetInstance                  gc      - GetClass

     ci     - CreateInstance               di      - DeleteInstance
     mi     - ModifyInstance               eq      - ExecQuery
     gp     - GetProperty                  sp      - SetProperty

     tms    - TotalManagedSpace            tp      - Test pools
     ecap   - Extent Capacity              pd      - Profile Discovery

     im     - InvokeMethod                 active  - ActiveControls
     ind    - Indications menu             tv      - Test views

     st     - Set timeout value            lc      - Log control
     sl     - Start listener               dv      - Display version info
     ns     - NameSpace                    vtl     - VTL menu

   
   

     q      - Quit                         h       - Help
   ########################################################################
   Namespace: root/emc
   repeat count: 1
3. Run the command eq at the prompt
   (localhost:5988) ? eq
   Query Language[DMTF:CQL]:
4. Run the below query
   Query []: SELECT EMC_ArrayChassis.SerialNumber FROM EMC_ArrayChassis

A working provider will return the following for each disk system attached to this provider:

++++ Testing ExecQuery:  ++++

Instance 0:
ObjectPath : //10.0.10.54/root/emc:Clar_ArrayChassis.CreationClassName="Clar_ArrayChassis",Tag="CLARiiON+CKM00083900053"


Clar_ArrayChassis

CLARiiON+CKM00083900053

CKM00083900053

Number of instance qualifiers: 0
Number of instance properties: 3
Property: CreationClassName Number of qualifiers: 0
Property: Tag Number of qualifiers: 0
Property: SerialNumber Number of qualifiers: 0

ExceQuery 1 instances; repeat count 1;return data in 0.000000 seconds

Retrieve and Display data - 1 Iteration(s) In 0.062400 Seconds

A failing provider will return the following:

++++ Testing ExecQuery:  ++++

Error: Connection closed by CIM Server.

Retrieve and Display data - 1 Iteration(s) In 0.053000 Seconds

Please press enter key to continue...

Securely eraseing disk and file

 I used Hiren's Boot Disc and DBAN boot disc to erase my HDD securely most of the time.  I did use dd command but found out there were few other useful commands recommended in this link.  

The first one mentioned in the article is quite useful.  It will overwrite the disk five times and add a final overwrite with zero to hide shredding.

 # shred -n 5 -vz /dev/hda

The command below will use shred and /dev/urandom as the source of random data.

 


shred -v --random-source=/dev/urandom -n1 /dev/sda

 

Another option is to install scrubbing program like scrub in Unix.  You can use it to erase a single file with algorithm.  

scrub -p nnsa|dod|bsi|old|fastold|gutmann|random|random2 fileNameHere

 

Or you can even use it to erase disk with dod standard.  

scrub -p dod /dev/sda

 

*** WARNING: Please test the commands before running it on your data.  Commands mentioned in this article will delete the data permanently.  *** 


  

Installing VMware Workstation Pro in OpenSuSE LEAP 15.4

Now system is up and running fine.  Next is to install VMware Workstation Pro.  Basically, it is my OpenSUSE LEAP 15.4 is my host. 

Make sure system is up to date and install the required packages.  Download the binary and copy it to a tmp folder.  Then add the execute permission to the bundle. 

sudo zypper update
sudo zypper install kernel-source kernel-devel gcc

Download the binary and copy it to a tmp folder.  Then add the execute permission to the bundle.   Run the installation after. 

sudo chmod +x VMware-Workstation-Full-16.2.4-20089737.x86_64.bundle
sudo ./VMware-Workstation-Full-16.2.4-20089737.x86_64.bundle

Reboot.  Then I receive the errors below one by one.  

 

  

Do some researches and follow instructions in this video.  

First get the VMware version
vmware -v

Then, download the patches needed to build the VMware host modules against the recent kernel.  You may need to rebuild it if VMware does not start after update.   See https://github.com/mkubecek/vmware-host-modules

wget https://github.com/mkubecek/vmware-host-modules/archive/workstation-16.2.4.tar.gz

tar -xvzf workstation-16.2.4.tar.gz
cd vmware-host-modules-workstation-16.2.4/
tar -cf vmmon.tar vmmon-only
tar -cf vmnet.tar vmnet-only
sudo cp -v vmmon.tar vmnet.tar /usr/lib/vmware/modules/source
sudo vmware-modconfig --console --install-all

I setup a script to start vmware every time the host boots up.  

/etc/init.d/vmware start

I found out none of the USB devices list under Removable devices of the VM.  So, I suspect the vmware-usbarbitrator is not started up auto.  Add the following line to the start up script.  Now, I can see the USB device.

/usr/bin/vmware-usbarbitrator start 

One issue I found and other ppl complain in other Linux distro too.  If you try to copy from VM and paste on the Linux host, you will get error below.

There seems to be no solution to it.  However, if you check your /home/user/.cache/vmware/drag_and_drop/ directory, you will actually find the file there.  

OpenSUSE LEAP 15.4 as physical host

There are still some issue with USB3 for Debian / Ubuntu on my old Gigabyte GA970A-D3P.  I decide to rebuild that with OpenSUSE LEAP 15.4.  I don't need a lot of power.  So, AMD CPU FX8320 I got back in 2013 is good enough.  I follow the link below.  It is self-explanatory.  

https://doc.opensuse.org/documentation/leap/startup/html/book-startup/art-opensuse-installquick.html

For NIC, I just set it up for IPv4.  Enable firewall and disable ssh.  Keep SecureBoot to test Windows 11 as VM in the future.  I don't like the GNOME desktop and it looks like a table to me.  So, I select Xfce instead.  Since it is a physical host, I remove tools I don't need like SAMBA (if need to share files, I will use USB drives instead), Thunderbird, Transmission and Pidgin.  So, I remove them. 

Installation runs fine.  nouveau is used for the NVIDIA GT630 and Realtek 8812BU WiFi adapter is not working.  

So, I decide to use NVIDIA driver instead of nouveau driver.  After installing G05 driver, the resolution drops to 640 x 480.  Exactly as what's described in this post below.

https://forums.opensuse.org/t/leap-15-2-nvidia-driver-does-not-work/141453

I decide to go with G04, and everything works as expected.  I don't follow the instructions found in https://en.opensuse.org/SDB:NVIDIA_drivers. 

Instead, I go to https://opensuse-guide.org/3d.php and install the GeForce 400 series drivers.  The 1 click option will add NVIDIA repository.  Once installation completed, reboot and open NVIDIA X server settings to confirm installation is successful.
 

 

 

 

 

 

For 8812bu adapter, since mine is USB, I run lsusb to confirm the model number. 

Then, follow instructions in https://github.com/morrownr/88x2bu-20210702  If driver fails after system update, please follow instructions in the link to fix it.   I copy the steps from the link. 

1) confirm system is up-to-date by running

sudo zypper update

Then, reboot after update completes.

2) Install required packages

sudo zypper install -t pattern devel_kernel dkms

3) Create a directory to hold the downloaded drivers and move to the newly created directory

mkdir -p ~/src

cd ~/src

4) Download the driver

git clone https://github.com/morrownr/88x2bu-20210702.git

 

5) Move to the newly created driver directory

cd ~/src/88x2bu-20210702

 

6) Run the installation script but it complains about missing iw package.  So, install iw package first then run the script.  

 

 

 

sudo zypper install iw

sudo ./install-driver.sh

7) Reboot

There is a warning about kernel update.  If you decide to upgrade to a new version of kernel such as 5.15 to 5.19, you need to remove the driver you have installed and install the newest available before installing the new kernel. Use the following commands in the driver directory:

$ sudo ./remove-driver.sh
$ git pull
$ sudo ./install-driver.sh

For the 8812BU WiFI, I see another similar link https://www.fastoe.com/blog/install-rtl8812bu-usb-wifi-dongle-on-linux with driver build and installation steps. 

smartmon tool is installed during the SUSE installation.  So, I just need to run the smartctl -a /dev/sdX to look for any disk errors.  Or install gsmartcontrol if you want a GUI.

I have an USB drive to store my documents and pictures.  On the safe side, I have a 2nd USB drive as a backup of backup.  To sync them up, Grsync is used.  

When I open the yast2 firewall, I do not see the wireless adapter listed.  So, I wonder if it is protected by firewall.  Going thru the documentation, firewalld replaced SuSEfirewall2 in SUSE 15.  I run the command below to confirm the wireless adapter is assigned to the public zone.  

firewall-cmd --get-zone-of-interface=wlan0