How to complete my graduation after a break?

It is nice to see your interest in continuing your graduation after a break. I think universities set maximum time limit to complete the course, say 6 years from the time you enroll into the course etc.. So you can better to go to the college and clarify your doubt with them, if it is possible to continue the course, that is to clear the backlogs.
If that is possible then you will have to pay the exam fees, and sit for the exam and clear all the backlogs.

What I need to do if it is not possible? I need to apply for fresh degree course right?

Almost 13 year have been passed since you have completed B.Sc. Maths through 2003-2006. I think university will not allow you to take arrears after such a long period. Because, during this long period lots of changes may undergone and it might be possible that all syllabus and pattern had been changed. But you can request to university in writing. Rules varies from university to university. Might be they allow you to appear on the examination. However, chances are very less. But you can give it a try. If they would allow you will be able to take pending papers and get your degree.

The syllabus and other regulations of the course might have changed many times from your studies year to now. Generally, you have to complete your backlogs within 2 years. Under special conditions, they may consider for some extra years also. But in your case, it is 13 years which is very high. So no possibility to complete the arrears now.

You have to take a fresh admission and study all the subjects again. You have to write all the papers again and you should obtain minimum marks as per the rules and regulations of the university.

So please don't have any hopes for completing the arrears and start studying fresh. All the best to you.

As per the existing rules and practices the arrears are to be taken up and completed within 2 years for normal graduation and 3 years for engineering courses. As per that you must had cleared them by 2008-09. You might have reasons for could not do so and sometimes the reasons can be genuine also. There is a special relaxation given in specific cases but for that universities have their own deciding board and you can take a chance there by applying for a one time relaxation and allow you to clear the backlogs. Please remember that in such cases they might take an exam to assess your potential for that and only after that they might allow you. There are some possible conditions where such relaxations are provided. For example if the father of a student dies and due to family responsibilities he joins a job then it is natural that he will not be able to continue his education. Those are the special circumstances when something can be done to help the student at a later period. If you feel that you have some genuine reason of that level then there is no harm in trying to apply for relaxation. Otherwise the only recourse is to go for a fresh degree. Take a fresh admission.

You have made up your mind to continue your graduation - course despite the fact that almost thirteen years have passed on relinquishing this course. Hence to have a detailed information whether, you can clear the backlogs at this stage has to be clarified by the Principal of the college.
It may so happen that regulation of the course - structure and the rules of the university undergoes changes time to time and moreover there is a time - limit to complete the backlogs mostly within three years from the initiation of the courses. You may have clarification of the prevailing rules from the Principal.
Apart from that there exists a meeting of the University officials permitting a candidate to appear for the backlogs on some special ground such as death of any earning member of the family resulting in discontinuation of studies as a consequence of taking jobs. In that way, the special case may be applied to you.
Lastly in absence of any concession, you have to start your course afresh.

You have a huge gap to complete your degree. What is the need to complete this degree after such a long time?
If it is required to have a career after completion, I would suggest you enroll fresh in a course in which you have an interest and have confidence to do good in the same.
After such a gap, no institutes allow completing any degree because of many reasons like age criteria, curriculum change, etc. Apart from these reasons, are you confident that if the institute will allow you to sit in examinations and clear your exams, will you be able to do that after this much gap? If not, I guess you should not waste on time on this and better should give a fresh start to your career by enrolling in a course suitable to you.
The final decision will be yours depending upon your financial and family suitability obviously.

Universities generally allow only 2 years extra time for clearing the arrears so I do not think that you can get a chance for clearing them in that way now. The better approach will be to join fresh as regular or part time as evening schools are also very common now a days. You can also think of doing it in correspondence mode also which is very convenient in case you are already busy in a job.