It's been done. I think it was Gambit Scheme that had its own bignum library. And for a while, very large integer arithmetic was reportedly faster than GMP, which if you know anything about GMP is quite an achievement. However, the GMP guys subsequently fixed this problem.