Category: Words

I always forget the basics

Jeff Miles Leave a comment

I had a strange issue with one of my branch offices, where they would lose access to local resources and external Internet sites whenever our Site-to-Site VPN with the head office went down.

I spent around 3 hours troubleshooting this issue, desperately looking for a logical cause. It wasn’t until I paid closer attention to the DNS settings that were being received from the DHCP server did I notice that the primary DNS nameserver was a legacy domain controller within the branch office that no longer existed, and the secondary DNS was a domain controller in our head office, across the VPN.

When the VPN link went down, the clients had no resolvable DNS servers, and thus couldn’t access anything except by direct IP.

When I discovered this, it was a quick fix that brought services back online promptly.

Unfortunately it is all too often that I dive into a problem looking for a cause that is complex without seeing the simple issue right in front of me. I need to learn to be a little more methodical in my problem solving, and start with Layer 1 first.

Well that was unexpected

Jeff Miles Leave a comment

I’Road_ThumbsUp_Successm working late tonight fixing some stuff with Sonicwall, and it worked on the first try! Now that my SSL-VPN is configured over port 443, hopefully it will pass through a client’s super restrictive firewall without issue and solve a long-standing issue for my user.

I’m calling it a night, ending on a high note.

IT Operations vs IT Projects

Jeff Miles Leave a comment

Does your IT department understand the different between IT Operations and IT Projects?

It has taken a while but I’m beginning to understand the necessity of separating those two concepts, with the goal of being able to excel at each.

 

The IT Department at my company has grown, and while statistically it has kept pace with the growth of the company, realistically when looking at staff utilization and slipping deadlines there is definitely room for improvement.

When I started in 2005, I was the second IT staff member. Between my co-worker and myself all IT responsibilities were handled; from user support up to infrastructure planning, purchasing and maintenance.

It wasn’t until January 2012 that we added and additional staff member to assist with IT Support; by this time the company was pushing 400 employees across 6 locations.

Due to the fact there were only 3 of us, there were still a large amount of shared responsibilities; despite looking after all of our infrastructure I never hesitated to assist with a minor excel problem. And for a company that only has 3 IT staff members and isn’t growing that may not be a problem.

However now in 2013 we’re at 5 people (two IT Support, one Jr System Admin and 2 System Admins) and are beginning to see the error of sharing responsibilities when it crosses the concepts of IT Operations and IT Projects.

Inevitably, IT Operations takes priority; if your revenue generating department demands an upgrade to the latest version of AutoCAD, and it’s approved (or already purchased), chances are effort is going to be made to complete that task. If the person responsible for that task is also trying to implement some new project (like FTP site replacement or Exchange upgrade or Hyper-V management, etc) it is highly likely the IT Project is going to be pushed back.

For IT Operations to be excellent, those staff need to focus on IT Operations. For IT Projects to be excellent (on-time and on-budget), those staff need to not be distracted by user support and install requests.

At the same time, I would say there still needs to be some overlap in one direction. Those working in IT Operations may benefit from inclusion in small IT Projects; to challenge their skills and allow for growth, learning and mentorship. It is motivating to be given some responsibility that is more than break-fix work.

 

Workload and career direction

Jeff Miles Leave a comment

2013 has been a ridiculous year when it comes to projects and priorities for me at work. It seems like the workload never lessens as I make progress on things, but rather keeps increasing as more opportunities for improvement or upgrades are found.

The majority of my time has been taken up with software development of an internal line-of-business application (using ASP.NET C#). Not just programming but information gathering, analyzing business rules, defining requirements, coordinating with an external developer, writing SQL scripts to point the developer to the right data and programming to finish the application off.

I have genuinely enjoyed this project; it’s not really often that one in IT gets to ‘build’ something, especially something that will be a cornerstone of the company’s processes and procedures. However if you had asked me a few years ago whether this would be part of my responsibilities, it would be one of the furthest things from my mind.

One of the problems with building skills in this area and devoting so much time to it is that there is always just one more site to build; one more business process that can be improved with a little bit of database and interface magic. Which is exciting but ultimately not what I want to be doing for the next 10 years.

The more I think about this subject and my career path, the more I have come to realize that I’d really like to get back to IT Projects. This would be implementation projects like Microsoft Exchange, Citrix XenApp, disaster recovery infrastructure and more. I have neglected this side of my duties simply due to time constraints, and I miss being so involved in my infrastructure.

 

These thoughts have been discussed with my supervisor, although a plan of action has yet to be determined. I feel better knowing that my future is less of a gray area, and I have defined the path that I wish to follow.

 

DFSR Crash and replacement search

Jeff Miles 2 Comments

A few weeks ago my DFSR database crashed, and crashed hard. I won’t go into the details of my troubleshooting steps, mostly because I didn’t make good notes while it was on-going and I was very sleep deprived. Suffice to say I spent many hours trying to resolve it, and wasn’t successful.

It was a Tuesday night that I disabled the DFS folder targets on my branch office servers, forcing all of my remote users to access our namespace over the VPN. I was hesitant to do it since our fastest link is 5 Mbps but it was the only way to ensure data integrity. Following that we needed to manually sync the data from our spoke servers to the hub since there had been 3 days of non-replication.

While that was going on, I began looking for a solution to our problem. Our problem is that we need to have those within our branch offices working on the same files as the head office. These are files used across a variety of applications including AutoCAD and ArcGIS, so our users are expecting fast access to data that can be quite large.

This is something that is difficult to find information on; not many people are talking about how they handle branch office file collaboration especially in a larger company.

In my case I tested PeerSync for a few days to see if it could replace DFSR however there were a few problems we encountered with our environment which make it unsuitable. In the end I re-implemented DFSR across our 2.5TB of data, and just waited for initial replication. This took another week.

Since then DFSR has been running smoothly, however I’m still looking for a replacement that will be scalable for my company as we grow in offices and data size.

 

Right now, I’m considering two options:

  • Remote Desktop / VDI
  • WAN Acceleration

Both would be a fairly substantial capital investment initially, but with the growth my company has seen it is inevitable.  It has been a long time since my last post because I’ve worked so much overtime lately with this issue in the midst of other projects occurring, and I just haven’t had the mental capacity to sit down and write.

In the next week or two I’ll be noting my thoughts on the two options above.