I can see how *Christian books* could be useful; however, there are so many [Christian books][1] out there; *Christian books* may just be too broad.  You might still find yourself having to explain and search for more specific adjectives that won't imply what you want to avoid. 

Let me draw on a little seminary training to provide you with some further suggestions that don't concede to "Testament" or "covenant":   

The 27 Books  
Primary Apostolic Writings    
Primary Christian Writings  
  

A bit more eccentric:  
The Christian Rule of Faith  
The Christian Anthology   

It might be helpful to refer to the collection by its components:  *Gospels, Acts, Letters and Revelation*, either list them, or refer to the particular part that is pertinent to conversation.  Monica may have mentioned that.

More specific concepts may be used of the original Greek documents, like the Greek Canon,  *Christian Greek Canon*, or any of the above, replacing *writings* with *documents*, *manuscripts*, *parchments*, or *corpus*.

A bit too obscure:  
[*Early Christian writings*][2]   
[G*reek Christian writings*][2]   
[*Apostolic writings*][3]

One might think using *earliest* instead of just *early* would remedy this; however, *earliest* has connotations concerning (carbon) dating of the manuscripts and if you merely intend to imply *foundation*, then, *primary* would be the word of choice, as *foundational* would be a mouthful.

I hope this equips you some to communicate more effectively about such matters.


  [1]: http://www.christianbook.com/
  [2]: http://www.earlychristianwritings.com/
  [3]: http://www.churchhistory101.com/century2-p2.php