Triple CCIE so what? – Passed CCIE SPv3
Well, I think the time has come for my blog post to hit the cloud re the journey to the world of triple CCIE. I have consolidated my third CCIE journey in this blog post. The purpose of this post is to inspire other candidates who are still on the road to achieving their CCIE. Most people think once they get their CCIE, they are the guru. I don’t think so, you don’t have to have a CCIE to be a master of a particular technology. I have seen many of my friends who do not have a CCIE but they are capable of writing an RFC and doing embedded design work. However, what CCIE really teaches you is discipline and thinking outside the box and tackling any challenges that you might see in the field and quickly come up with the right approach. Being a triple CCIE, I really think there is so much to learn and always there is a gap to learn new things.
Way back in 2004, when MPLS was quite new, I was involved in deploying a few small MPLS networks. The largest one I deployed was a 7-nodes MPLS network that covered the whole country. Wheee. 7-nodes covering the whole country, a bit of a surprise here? That’s true, yeah, it was a small island country in the south pacific, where Telco normally do not have many customers. I am talking about way back in 2005-2007. I love working on new things and in the past I had an opportunity to work on N7K/ASR(XR). XR is totally a new beast in the CCIE SP lab but not new for me. Don’t be afraid of XR as it has slightly different command syntax than IOS. Afterwards I had an opportunity to work on different technologies (voice, datacenter and big data). Recently I have been involved in a project that would not need more of my extra brain cells and I had plenty of (outside office hours) time to do and try something “new”. I was not thinking for another CCIE at all but instead I started thinking about porting a customised version of Openflow on a netFPGA board.
I bought 2 boards (bloody expensive though) and started building a list of *nix lib and packages that I’d want to make my first netFPGA board working. The idea was to build a small embedded OS that can port OpenFlow. I started working with buildroot to build the embeded OS and spent nights and nights in coding and preparing the Makefiles to be able to do what I wanted to do to port OpenFlow. My OS poting was completed and the netFPGA board was running on my own customised embedded OS. I had a few issues with the Openflow libs, the interfaces wouldn’t come up. I was then running the latest Linux kernel v3.8. I went back to the buildroot again and re-compiled the whole libs and right kernel 2.6.32 and ported it on the netFPGA board. This time bingo..I was so happy as I could see my whole thing in ACTION, interfaces were up and working as I was expecting. ) Whooooooooooooo!
To give you a few ideas of what a netFPGA application can be, have a sneak peak at the image below:
I was enjoying the journey of building my own switch based on netFPGA board. BTW, I hated rasberry pi boards as there was too much work and vendor wouldn’t provide a lot of libs/firmware that is required to do something new!. Anyway… oh no, I thought I was writing a blog post for my CCIE journey, sorry lets go back to it.!
I changed my job in 2012 and was playing tennis. It was so much fun as I switched to tennis from cricket. As I was new to tennis, I forgot everything about my backyard netFPGA project. The fact was it was too time consuming. To get something working, I had to play with script/codes for ages. I don’t write codes on a regular basis so even trapping a mistake in a few codes logic, it was taking a lot of time. It is not fun when something is not working as what I had initially expected. There is not much support that one can get from the Internet. One day I was doing some reading on BGP and it eventually got interesting when MPLS-TE came along with the above diagram that was using the netFPGA for Optical networks. I thought this is it, I wanted to learn and master MPLS BGP. Then I thought should I go for SP CCIE. I just wanted to challenge myself so decided to go for SP CCIE track. I did some research on the Internet on SP lab and found out most people on blogs were talking about doing R&S before the CCIE SP. I just ignored them and decided to take the hard bullet first.
1.2 INITIAL PREPARATION:
- I downloaded CCIE SP lab techtorial from CiscoLive site and started reading them. I went through all the slides and then I bought some MPLS TE books written by Azhar Sayeed. Monique Morrow . A friend who failed SP lab 3 times and unfortunately he finally gave up! (what a shame!). He was happy to hand out free IPexpert/INE CCIE SP V2 video tutorials. I went through them as well. They were awesome in terms of understanding the technology and the goachas. The INE workbooks were outdated but had very important tips and tricks. It didn’t cover the XR part though.
- Awesome custom labs from CCIE SP program manager (a must have) – Vincent Zhou
1.3 PREPARATION TIME :
It took me almost 6 months to pass the lab- This is from start to finish. The Detailed breakdown is below:
- CiscoLive SP Techtorials – 1 week
- MPLS TE book – 1 month (was reading ’em on and off)
- INE/IPExpet Video – 2 months
- INE/IPExpert workbook practice – 1 month (only done the last labs but went through all the labs to see what they were trying to simulate)
- Vicent Zhou’s custom labs – 3 weeks (this is a must have, I repeat, this is a must have material)
- Tech notes and docCD – almost most of the time when I needed any clarification on something that was new to me.
- Improving lab start-finish speed – 2 weeks full time labbing.
I am not a plug for INE/IPExpert but they do have some awesome info that I couldn’t get any where else.
1.4 CCIE SP HOME LAB:
I spent almost a month in buying a proper server for my SP home lab. It happened someone in Cisco (internal) wrote a script for INE workbook practice in a simulator. It was such a time saver for me, as I didn’t have to write IOU/XR topology files to practice the INE labs. Thanks Brad R (A great guy in Cisco Advance services:). It took me a while to make the simulator (what is it, how can i download it ?) work in a way that I wanted it. But after initial hurdles it worked like a charm. I was able to do most of the lab tasks with this magic simulator. For any few features that the Sim couldn’t do, I used our internal company’s lab based on real hardware i.e. beefy XR routers, ME 34xx switches and other IOS routers. Please do not ask me about the simulator as this is internal and Cisco partners only at this stage.
Here is Vincent’s sample lab topology:
1.5 WORK/FAMILY LIFE and TIME MANAGEMENT:
I was lucky enough to work in a good team environment and as a result I didn’t need to work after hours to resolve customers’ issue(es). In our team, all members are CCIE (double/triple) and are pretty switched on. A few guys in the team are like a walking encyclopaedia, you just ask and save time in googling/labbing. That means you spend less personal time to solve customers’ issues and more time for study (umm). I know a few guys who have got such an understanding manager that allowed them to study while at work :). Is there anyone else who has this kind of understanding manager? I wish I was one of them but unfortunately my work commitments didn’t allow me to have such a luxury. However, I developed the following strategy that worked well for me and I think that might also work for some of you dudes lurking out there :):
- Organise and tidy up your work – for example, if you have 10 tasks to deliver this week. You can sit one day at home and work on all these 10 tasks together. at least you may complete them 60-80%. You can almost do a week’s worth work in less than 16-18 hours (am I crazy in saying this?). You can keep these work related tasks handy. Keep them ready in your pocket just like candies. When your Manager asks to deliver one of those pending tasks, you just smile and an hour later hand it out to your manager just like candy. This is just an example re how you can save your brain cells and balance out your work. This will also give you a true sense of 9 to 5 job and you’ll have some room in your brain cells for study. The idea is, the less office work related tension you have in your brain, the more productive you’ll be in observing the study juice. If you’re job is like replying to hundreds of emails on a daily basis then you might need to come up with email strategy. This is just another skill to learn but it is very useful to save time. Many of you out there would know what I mean. CCIE and crazy work schedule do not go together . Email management: Most of us do this but may be useful for those who are not doing it yet. I know this has become an old trick now. Use office Email on your mobile and answer them while you are commuting. This saves a lot of time. Many work related tasks you can do while you are not at work. for example, if you are waiting for an input from customer you can email them 11pm and they will usually reply to you before you get to the office in the next morning. It’s good for everyone. saves you a lot of time waiting on a customer.
- Weekend fun/activities. – Give it up for a while if you can, no more drinking with friends, I gave up playing tennis and going out. No more bush walking either. Social life – Forget about this, not going to happen. CCIE and social life do not go together either. This is completely legit to tell your friend that you’ll be busy and do not appreciate their offer to go to a pub and enjoy one of those late night outings :(. A good friend will understand that. I guess, it would be hard to negotiate with your wife!. Exercise: Everyone knows, sitting in front of the computer is not good. You must take breaks now and then. Between the lab break, go out and do some exercise. If you’ve a swimming pool, that is the best. A 30 minutes swim or 25 minutes run is more than enough. (BTW, I can swim now) Typical thing I’d do during the
break was clean up the house. Wash the dishes in the kitchen, do dusting, vacuum. anything that is moving your body. I think these easy activities chew up a lot of calories. Obviously, eat healthy food during study as the last thing you want to do is to get <sick>.
1.5 LAB PRACTICE STRATEGY:
I have used my own home lab using special simulator in VMware environment :). It was very handy as I could just do a part of the lab and (if I want to ) have a power nap and then resume the lab without worrying about time running out. Unlike online rack rental we have to finish the lab within the time constraint. Unfortunately CCIE SP lab can not be “FULLY” simulated (XR Part) with the material that is openly available on the Internet. On stolen IOU and freely available dynamips/GNS3 you can do some practice. An alternative could be you can build up SP lab based on stolen IOU and dynamips/GNS3 and when lab date is near then hire online rack rentals.
Last 2 months of my journey I used to practice everyday, if not alternate days with different workbook. You must go through Vincent’s labs. Without them it is not possible to pass the lab, unless you design and deploy MPLS a day-in and day-out basis. A guy who is reading this post and been to the lab before must be smirking re what I am on about. He knows what I mean. Anyhow, you’ll know what I mean once you go to the lab first time and then see Vincent’s labs later on. Without breaking the NDA, I’ll give you an idea what gear an actual CCIE SP lab has:
My typical study and labbing time was:
- Monday to Wednesday – study, watch videos, browse through tech notes.
- Thursday – resting.
- Friday Saturday and Sunday – extensively labbing. I studied part time with my FULL time job. For some people it could be challenging to get time for study. But hey, if you really want to have a drink, u will find a pub no matter how far u got to drive. The same thing applies to the study and “MAKING” time for it. If you are serious make time for it :)= my plug.
I’d spend most of the time in study and less time in practice. Later stage (near the lab date) it was reversed -Most of these hours I’d spend in practising, checking notes, summarising the task list and updating my to-do list. I documented everything that I encountered during the practice labs. I focussed on straining my brain cells on why this way the solution is working and not the other way.
I initially thought to use alias in IOS/XR to save some time but once I did the lab 10 times, all commands that were required to verify the tasks were straight in my head and I could type them quicker than remembering the alias and typing them out: for example: top 10 commands:
show bgp ipv4 blah.
show ip ospf int bri,
show ip igmp ,
show mpls int,
show mpls ldp discovery
show mpls ldp nei
show mpls traffic-eng tunnel
show ip pim rp
show ip pim nei
show ip pim int
show clns protocol
show clns nei
show isis nei|int (on xr)
1.6 FIRST ATTEMPT EXPERIENCE (JAN’13 SECOND WEEK):
Being based in Sydney, I was able to get SP lab date easily. In fact there are not many people taking SP lab here. Probably it’s too boring for them while seeing the other nicer cakes a.k.a Bigdata, Wireless and DC lab just out there. I booked the lab for the second week in January, 2013. I was feeling 90% ready as I have been through the MPLS MBGP, MCAST, BGP and L2/L3 VPN technologies. I repeated the videos and re-viewed/practised the vendor and vincent’s workbooks. Nothing was left that I could think of but I was excited to go ,sit and experience another lab game for the third pie!
2 weeks before my lab date, I took a week’s break from work and focussed on speed and summarising the tasks and the right order to implement them. I was doing labbing 16 hours a day and a few hours of revision. Day before the lab, I went to bed at 11pm. I had a few glasses of red wine. I woke up in the morning 15 minutes before my alarm bell rang. Got ready and in 20 minutes time I was on the way to the lab. It only takes me 35 minutes to get to CCIE lab in Sydney by train. I know a few folks have to travel from overseas and interstate but for me it was just like another commute to work.
In the CCIE exam center, we were 4 guys (unfortunately, no ladies), 1xSP candidate, 1xR&S, 1xVoiec and 1xWireless candidate. Nobody was doing storage/DC. After a quick briefing, the proctor showed us the POD#. I was very impressed to see dual monitors (I wish it would stay this way) on my POD. The SP lab exam has no papers like we did for Voice/Security exams. I opened the question paper and diagram on the right hand side monitor and then opened up the sessions to all devices on the left hand side monitor. None of the other CCIE pods had dual monitors so I was already feeling like a very SPECIAL frog (see frog in the left, jumping frog from the Rack). I think somebody had well thought out about putting dual monitors for SP lab candidates. It makes complete sense to have dual monitors for SP lab. The matter of the fact is that there are so many devices in the SP lab that need a separate monitor. Jumping between the routers is a waste of time. I read the whole lab in 45 minutes, noticed there were a lot of questions that I didn’t know the answers. I just made a quick summary and diagram in mind. I didn’t draw it because lab diagram is already there on the screen why bother duplicating the same thing. I made a list of important parameter for IGP, iBGP, eBGP peers, interfaces that need to be enabled for LDP/PIM and then started doing tasks in the order they were presented in the question paper. I skipped a few tasks because I didn’t know the answer as I said before. Then just before lunch, my IGP started playing up. The RouteReflector router would show up in the routing table but somehow the whole BGP topology was broken. I thought, right on, here we go again the cat and mouse game is on for the third time. I said to myself I have seen this before so I was calm and was going through the show and debug commands to trap the fault. I managed to fix it and it turned out that this issue was with the hardware. Proctor was so nice and understanding, I explained to him what the problem was and he said, continue doing other tasks while he fixes this. Finally the proctor fixed the problem and gave me 30 minutes extra. How kind? well I really spent about 40 minutes to trap this fault so why shouldn’t I get it in the first place. As you know every minute in the lab matters.
Just before lunch I had IGP, e|iBGP, MPLS, PIM up and running. I had some issues with the ISIS, and one of the Mcast/MSDP questions I misread and implemented the wrong solution. I went back to the docCD and found the answer and commands that were required to finish up the task. Anyway..during lunch break everyone had quick lunch with proctor. Surprisingly, after 20 minutes everyone was back to their POD# again. I think the proctor wanted to finish early and go home and also the student wanted to get back to the lab ASAP. That quick lunch strategy works out well. After lunch see the the voice guy was sweating and bleeding, his phone was not dialing out anything as I could hear fast busy beeps. That reminded me of my first few lab attempts when I was doing my Voice lab. This guy must have been frustrated. Anyway, after lunch, I finished CSC, MVPN and l2vpn section. My CSC was broken and I couldn’t fix it. Eventually I ran out of time and noticed that proctor calling ‘times up’. It was 5pm when everyone left the lab. I got the train back home and was hoping I might pass. 2 hours later the bad news finally come, …….FAILED. I went through the lab score report and found that I did a bad job in VPN and IGP sections was not 100%. I nailed L2vpn and other sections. I just took it as a part of the learning curve and to come up with better strategy and be better prepared for the next time. I thought to come back after a month or so and re-attack the beast with a sharper sword. The lesson to learn here is (to the reader as well) after failing the lab, never put your journey off track. Get back on the horse ASAP and stick to it. Never ever think about giving it up (just like image in the left, frog can do so can we)! Immediately after I started looking for the lab date. I found another date and booked the exam for Feb last week. This time I thought I desperately needed a break from everything. I have been studying and working full time for more than 5 months. It happened we had a new team member who just joined us and was pretty switched on and I asked if he could fill in my role. He said no issue. After this, I got approval from my manager and I was on the mission of doing nothing but wanting to give my brain a thrill, challenging myself and a good work out and a true break from work and CCIE lab crap. The bottom line, never go the lab until you feel 200% ready. I thought I was ready but I wasn’t. I ran out of time, had no time for verification and tackling the lab jitters. Once you are 200% confident and ready, your CCIE# is just around the corner (Quoting Narbik here as these are his words).
1.7 BREAK FROM EVERYTHING:
I decided to do some adventurous sports so that I could give my brain cell a real thrill and challenge. I flew from Sydney to Queens Town New Zealand and did the following sports activity. While on-board in the plane this unexpected briefing of Air New Zealand made me feel that I was really going on a holiday to the middle of the earth! whoooooo. It was simply the most awesome briefing that I have ever seen on any aircraft. I wonder if that’s why Kiwis are different than rest of the world and do things differently.
Here is the link for the full airNZ video:
Those who have not been to Queenstown yet, you don’t know what you are missing. There are so many things to do. You’d never be a bro-dam! You can see below activities I was not sitting idle during my stay there.
- Day1 – Flying foxes. Flying from tree to tree. This was just like putting my feet into the tub.
- Day2 – 134 meters Nevis bung y. This really blew up my mind. For the first time I have done 134 meters.