Inside a Migration

Ms. Z. Arsenault is an IT consultant working in the depths of a large North American energy company. She’s one of those brave souls who works away in the background, keeping the servers running, making sure all the pieces fall properly into place so when the employees wander in each morning their applications run as expected. It’s often a busy job just keeping things on a steady path. But Ms. Arsenault and her team aren’t just maintaining the status quo, they’re also trying to improve performance and cut costs while maintaining a stable environment for the end user. This week I had the opportunity to talk with Ms. Arsenault about what’s she’s been up to in the depths of corporate IT.

OSNews: Let’s start off with a little bit about you and how you got into the IT field.

ZAZA: I was born in a small town from the east coast of Canada. When I was 18 and graduated was when I started trying to decide what I wanted to do in my life for a career. Computers were never a huge part of my life growing up, although the more frequently that I was around them, the more computers intrigued me. I decided to move to Calgary, Canada to go to a Technical Institute, about which, through friends of mine, I had heard good things. I graduated in 2001 with a Degree in Computer Sciences. After a few months from graduating, I worked in an entry level tier 1 Help Desk position, and eventually started in a tier 2 level Help Desk support role. After a few years of working there, I moved to a west coast city for a year where I worked in a tier 2 level Desktop position. A year later I moved back to Calgary, Canada and started work at an energy company, where I started on a combined tier 1 and 2 level support, with quite an open policy on supporting the desktop until the issue resolved. I had learned a lot in this role, as most of my job was hands on hardware support, which I had never had too much opportunity in my previous roles to dig right into. After a couple of years, I got word of a tier 3 role for Web Application Server Support/Administration. This goes hand in hand with the Java development I was taught in school, so figured it would be a great opportunity to venture into, and that is where I currently work. And so, here I am now, talking to you, and discussing my IT experiences.

OSNews: That certainly brings us up to speed. Please tell us a little about the environment you currently support.

ZA: Well I work in a large energy corporation, so the environment is quite large as well. There are approximately four hundred production servers, and possibly a thousand or more servers when you include testing environments, in this company. That being said, due to some users requiring more than one desktop at their desk, I could safely say that there are probably close to four thousand desktops, maybe more. My team of six, alone, supports around 40+ servers. I work in the Web Infrastructure department. There is a mix of Windows PCs as well as Unix/Linux desktop systems in the company’s production environment.

OSNews: What has generally been the most challenging aspect of running such a
network?

ZA: The most challenging aspect of running such a network is getting certain parts of the business to adapt to newer/supported versions of JDK. This would mean migrating their applications to said new versions. They understand the logic of migrating but they are just never ready to adopt these changes even though they say they understand the ramifications of keeping older unsupported JDKs.

OSNews: You mentioned (pre-interview) your organization is looking at making a
move to Linux servers. What operating system have you been running and
what prompted the change?

ZA: It was time to upgrade a good number of our applications to a higher JDK which is not supported by the version of Web Application Server I was using. Ever since before I started with this organization, they had a mix of operating systems, ranging from Windows 2000 Server, 2003 Server, Solaris, and Red Hat. To keep a consistency in our environment, I felt a need to decide upon a single operating system to run our environments from. This helps with some automation I have in place for log rotations as well as a front-end GUI I had built as the “middle man” between our developers and deployers.

OSNews: Which brand of Linux is the company looking to implement and why?

ZA: Most of the servers I have built for are being shared by multiple applications so I require a hearty and stable operating system. With that in mind, I have decided on 64-bit Red Hat Enterprise Linux 5 (kernel version 2.6.18). The servers usually have 2x CPUs and have around 8GB of RAM. I try to get away with 50 GB of disk space. I have jobs that run, which regulate any of the disks from filling up with logs, by archiving them. I will usually build a web application server on these boxes with either
Tomcat 6.x for simple Java applications, or for more of a heavier load of applications containing EJB or Cluster requirements, with Weblogic 10gR3. When Oracle bought out BEA, costs for the use of Weblogic in production environments rose quite high, so I only use it when I feel it is necessary to, and not as our primary application server.

OSNews: Will a change in operating system also mean rolling out new hardware, or will you try to keep the same servers?

ZA: Some of the servers I have in the environment are physical servers, and some are really old. I have not been keeping those for the migration, unless necessary to. Generally I have been using virtual machines using VMotion for many of our shared application servers. Also, the high costs of Oracle products is why I have chosen to use Tomcat for any non-cluster-required apps, as currently I don’t have many of them.

OSNews: What has the learning curve been like for the IT staff? Is there a lot
of new training involved to handle the new OS, new applications?

ZA: There is always room for training for anyone. I try to document everything in enough detail that even a newbie could jump right in and start operating the environment. I take pride in my documentation skills, because I know from experience that individuals with a strong IT background and a strong documentation skill-set, are sometimes hard to find.

OSNews: If we look at the total cost of the old system verses the new, is there a benefit of one over the other?

ZA: I decided that virtual machines (unless a physical machine is absolutely necessary) would suffice for our new servers, and in turn this decision helps the company with saving quite a bit of money on hardware. This also helps with the ease of scalability, more so than a physical server.

OSNews: Ideally, users don’t see any (negative) aspects of a change. What sort of steps/tests are the IT team taking to ensure a smooth roll-out?

ZA: When performing a migration, I have a strict process in place with a multi-tier environment, as well providing development, testing, and production lines. I always build a development environment with the same operating system and configurations I would have in Testing, as well as Production, but still with the flexibility to try new things. It is not necessarily a proof of concept (POC) server for the developers; I actually have one of those as well. They do still need to run any operating system or third-party requirements/additions by me before moving forward. I have documentation available to developers explaining the environments, and what we will provide for a service, where the logging will be, and how I prefer that they configure their data source access, etc. That way, there are no surprises and I have a consistent environment. It really adds ease to the transition. All developers develop differently. If I provide guidelines, from past experiences with successes and failures, then they may learn new things too. That being said, I am always up to trying new things as well if any are brought to the table, but this would be by starting in our POC server.

OSNews: Has your team run into any challenges with a heterogeneous (and thus complex) network?

ZA: When I first started on this team, there was horrible documentation and no structure. It was extremely challenging for anyone new to just jump in and understand the environment, because it was everywhere, and not everything was clearly documented. This has gotten better with time but the organization, as a whole, is implementing more of an executive focus on each I.T. team’s documentation by setting up a “timer” which flags documents older than a few months. This is helping with making sure that I keep up to date with my documentation by writing up Runbooks for my environments, for example. The organization’s Documentation QA is going under some great improvements so it will just get better.

OSNews: Is your organization considering a similar migration for the company’s workstations?

ZA: I’m not really sure, this is a department of which I am not part of anymore. If anything, the desktop team just makes sure to simply provide the user with an upgrade to the newest supported and approved desktops on an individual basis if issues are encountered which may require a substantial amount of time to resolve.

OSNews: Is there anything else you’d like to share? Are you working on a project in your spare time?

ZA: To add to my last answer about desktop migrations… once older computers fail, encounter software issues or are just simply unsupported and old, my company donates what they can to charities and schools and if not, then recycles.

As for other projects outside of work, I don’t have any at this time. I am quite comfortable here, but now that the environment has a better structure and consistency, for some people it could get boring… If moving to a new client to spice up my work life is what has to happen? So be it, it will just keep me on my toes. That is an important quality in a person within the IT industry, right? Being open to change!

30 Comments

  1. 2010-04-14 9:59 pm
    • 2010-04-14 11:27 pm
      • 2010-04-14 11:40 pm
        • 2010-04-15 7:47 pm
          • 2010-04-16 4:25 pm
          • 2010-04-17 1:14 pm
    • 2010-04-15 1:25 am
      • 2010-04-15 5:14 am
        • 2010-04-15 5:39 am
          • 2010-04-15 7:28 pm
        • 2010-04-15 5:39 am
      • 2010-04-15 2:22 pm
    • 2010-04-15 5:42 am
    • 2010-04-15 3:26 pm
  2. 2010-04-14 10:25 pm
    • 2010-04-14 10:29 pm
  3. 2010-04-14 10:31 pm
    • 2010-04-14 10:56 pm
      • 2010-04-15 2:00 am
  4. 2010-04-15 2:42 am
  5. 2010-04-15 10:08 am
  6. 2010-04-15 10:37 am
    • 2010-04-15 12:42 pm
    • 2010-04-15 3:31 pm
  7. 2010-04-15 2:25 pm
    • 2010-04-15 2:53 pm
  8. 2010-04-15 4:55 pm
  9. 2010-04-16 5:55 am
  10. 2010-04-16 10:24 am
  11. 2010-04-16 4:28 pm