The trouble with hacks

/rant

I want to rant. I’ve been working as an IT/Sysadmin for about 2 years now and there are two things that I am haunted by.

  1. DNS
  2. Group Policies

Now, I am always learning and I am by no means an expert at windows systems administration. I took on more and more responsibilities that removed me from the ‘IT Support’ role and let me grow into Systems Administration and I continue to learn daily.

Now, not get into specifics, but taking over a AD infrastructure that was neglected by hacks is terrifying. I refer to hacks, as in people that neglect the network, that don’t have a proper vision for documentation and structure and that don’t understand how AD and GPOs work.

Within the IT SysAdmin community “It’s always DNS” is a common phrase and a joke at times. Well god dam, I can’t believe how accurate it is or how powerful DNS is in a network.

You know what irks me? People that use crafty stupid hostnames for critical servers or any server at that. Stupid names such as “Sugar Baby” “Super Man” “Bat Man”, etc… you get the gist.

When you take over a network and have critical servers with stupid naming conventions like that, it can get very easy to shut down the wrong server or make changes because all of the names are so irrelevant. Especially when nothing is documented and you are left to your own to research and investigate carefully.

Not that I’ve had that happen, but I have had a mishap with a DNS record that was named something ridiculous. The server wasn’t even around anymore but a critical server was using that DNS record for a link to an IP in it’s hostfile. Something I never thought to check nor look into.

The other thing that annoys me is the ignorance of not knowing how to properly setup GPO’s and push them out to AD. You do NOT need to enforce everything. Stop doing that. After spending time looking around and cleaning up GPO’s, you wonder what would drive a person to just enforce everything.

Sure, if it’s a critical policy that you want in every OU regardless if it has Inheritance blocking or not but don’t enforce everything just because you are trying to push the policy out faster or believe that it will guarantee that the policy will get to the clients.

I cannot believe that a novice admin is correcting domain wide issue that a senior IT director of many years had made.

I can spend the rest of my afternoon ranting about stuff that I’ve come across but that’s not the point of this post. I wanted to get DNS and GPO’s off my chest only.

I suppose you will find this in any job/career. People that want to take initiative, drive, pride in their work and do the best with what they can. Others will just let things fall into disarray and not bother.

/rant

VMware ESXi – Cannot add VMFS datastore

To give some greater context, see my previous post.

When I was initially planning on how to setup these drives, I configured them with the HP P410 RAID utility as a RAID-0 array. I made the decision to not live such a risky lifestyle and blow away the array and configure it for RAID-1. I want to build a solid homelab that will assist me in aspects of systems administration so I didn’t want to risk everything by running the wrong array.

Anyways, when I booted into VMware, I was unable to add the VMFS datastore after setting it to RAID-1.

I received the following error:

“Failed to create VMFS datastore – Cannot change the host configuration”

As seen by VMware ESXi

I did a bit of searching around and tried to re-scan the datastore and get vmwre to detect it but nothing was working. I soon came across the following VMware communities post here, user Cookies04 was on onto something.

The user identified a very familiar scenario to mine.

From what I have seen and found this error comes from having disks that were part of different arrays and contain some data on them.”

That’s the exact thing that happened to me. RAID-0, some VMware data, then RAID-1.

I proceeded to follow the three easy steps and my issue was solved.

To correct the reported problem

I didn’t really have to post all of this but I wanted to in case somebody were to come across my page and had the same issue.

The interwebz if filled with many many solutions for issues. I’m just adding what’s worked for me.

🙂

HP Ml150 G6 – My first datastore

I don’t spend the amount of time on my home server as I’d like to. After a long day of sitting at my desk at work, dealing with production servers and everything super sensitive, I try to unwind a bit and work at a slow pace. My slow pace this week is my esx datastore.

I’ve spent the past couple of days thinking about how I want to setup the datastore that will contain my virtual machines. Initially I had the HP P410 RAID controller connected to two, WD Green drives in a RAID-o array. I was satisfied with that at first because the drives will run at SATA 2 speeds and hopefully RAID-0 will improve the performance ever so slightly.

Then I got thinking, my goal is to setup a ‘corporate’ environment at home. Multiple domain controllers, WSUS, Sophos Firewall, play with SNMP and PRTG monitoring but that made me realize that I don’t want to build a large environment that will go to waste if one drive was to fail. My ultimate goal is to move onto SSDs and use a more complex raid (RAID 6 or 10) for this server, but that’s down the line when I free up funds and more resources.

Last night, I decided to delete the RAID-0 array, pull out the WD Green drives and install two new-to-me 1TB SAS drives and proper cabling (Mini SAS SFF-8087 to SFF-8482+15P). I briefly talked about the cabling in this previous post.

I purchased a few SAS drives from ebay, not knowing exactly which one would be compatible with the HP P410 raid controller. Most of what I can find on the internet, points to the HP P410 controller not being picky with the brand of drives.

Initially I installed a two Seagate 1TB SAS ST1000NM0045 drives but the RAID utility would not want to see the drives. Thinking it’s the cable, I replaced it with a spare but the outcome was still the same. I did a bit of searching around and found a discussion on serverfault.com, regarding HP Proliant not recognizing EMC SAS drives. One user points out that some drives can be formatted in 520-byte sectors vs 512-byte sectors that you would normally get on normal PC/server class drives.

I haven’t tested that theory but I will. With that said, I decided to install two other drives, which surprisingly worked right away.

The drives that are functioning fine with the HP P410 raid controller are:

  • Dell Enterprise Plus MK1001TRKB
  • Seagate Constellation ES.3 ST1000NM0023

Now that I have two drive’s in a RAID-1 array, I loaded into VMware ESXi and proceeded to add a the new VMFS datastore. Adding the datastore gave me some issues, which I’ve documented here.

I have in my possession two SAMSUNG Data Center Series SV843 2.5″ 960GB drives that I purchased about 2 years ago from newegg for a fantastic price. I’ve toyed with using them in this build, but the SSD drives would only work at SATA 2 speeds. Maybe I’ll use them to house my personal data, but I should purchase a few more to do RAID-6 or RAID 1+0.

Regardless of my direction, I am still working out the kinks in my homelab environment.

Ideally, I’d like to find a cheap or reasonably priced NAS that has iSCSI ports. I then would be able create two datastores on the NAS, one for extended VM storage if required and the other for user data.

Thanks for reading.

Adding a vCenter 6.7 license

Hello, it’s me again.

From my recent blog post regarding setting up vCenter, I had difficulties locating the area to apply the vCenter license.  From what I found on the internet, it was referenced that you should go to the Host that contains the vCenter/VCSA VM, click on the VM and click on Configure. Maybe VMware changed it in version 6.7 but I could not find the same area for license registration under the VM itself.

Under the VCSA VM –> Configure –> Settings, I should see a ‘License’ section. I could not find anything of that sort.  I logged in as my admin account and my personal admin account, both that have the license role and that feature was still not available.

Frustrated, I did some looking around within the vSphere client and I found the area to do this.

You need to click on the ‘top’ FQDN vCenter identifier on the left hand side of the window, which houses your Datacenter and the nodes inside.

Once you click on it, you will see the following,

As you see, now selecting the VCSA and going to the Configure section and under Settings, we now see Licensing as an option. Now in my case, I’ve already applied the license but I’m going over where I went to do this.

You would select the Assign License button to proceed with entering your key into vCenter.

Under the Assign License window, you will have two options. To select an existing license or new license. You can import the license from your License section from the admin page or you can type in your license if you haven’t already done so.   I’ve already uploaded my licenses to the Administration License section, which I will show next.

Now what I have done initially was gone into the Administration section –> Licensing –> Licenses and typed in the VMware vCenter Server 6  Essential vCenter license key.  When I did this, the usage of the vCenter license was set to 0 and capacity was set to 1.  This was because I never assigned the license to the vCenter itself.  I did this in the Assign License window as seen above.

The last and final screenshot above shows the Administrator License window which identifies my License(s) and their state and capacity.

To note: When I was in the process of importing each host, the license for those hosts registered automatically here.  I did not have to enter the VMware vSphere 6 Essentials Plus License.  Those just followed with each host/node into vCenter.

My novice attempt are VMware maintenance

I’ll come out and say it, I’m not an expert or a confident user of virtualization and more specifically VMware products.  Over the last bit, I’ve taken on a more senior and technical lead position at my job and that involves more to do with the infrastructure side of things and not as much ‘customer facing’.  I’ve played around with VMware Workstation and Oracle VirtualBox but I haven’t done a hole lot in regards to ESXi, vCenter and the works.

I needed to ‘pull up my big boy pants’ and start learning as much as I can in the short time frame about our production ESXi cluster, trying to understand the configuration and anything that may be wrong with it.

When my department slowly withered away until it was only me, I’ve heard that our vCenter is broken and that management of the cluster is not possible.  Not having VMware support, I was really concerned about this broken system and how it would negatively affect our production and highly critical cluster.  I started doing some reading and came to realize that vCenter (VCSA) is only a central mangement feature.  Rather than using vSphere client to manage each invidivual node/host, vCenter allows you to manage the hosts all together (in a cluster) and enabled a few features, including High Availability (HA) and vMotion (allowing to move VM’s from host to host without downtime).

Knowing this, I spent any downtime I had reading up about vCenter and VCSA.  I looked at different installation methods (Windows vs Linux) the pros and cons of each.  vCenter can be installed on top of a Windows installation or it can be configured on a Linux machine and often referred to VCSA (vCenter Server Appliance).

My first question was regarding what vCenter/VCSA can I use with my cluster?   Luckly, I came across a page on VMware site that helps identify the version of ESXi and what version of vCenter is compatible.

With that sorted, I downloaded the most recent version of vCenter 6.7U1.  I choose to download the Linux installation rather than mess with Windows and use up a license for it.

Now with the .ISO downloaded, I searched high and low to find a good step by step guide on how to complete this install.  I already shut down the old vCenter VM that was previously created by our IT staff, which was having issued and filling it’s storage with logs.  Rather than try to troubleshoot it, I wanted to start with a fresh install.

I came across this fantastic link that helped me tremendously for setting up and installing my VCSA.  The notes and screenshots helped a novice like myself through this process.

As this was a live production setup, I was always fearful of something occurring but unfortunately I don’t have the resources to do it any other way.

Anyways, I felt that I wanted to share this quick post and the link to the site that helped me through this process.  Good articles go a long way in helping others out and that is one thing I want to focus with this blog site.  To provide good information that I discover or come across.

Thanks for reading!