CHARLESTON, W.Va. -- Joseph Lavigne Jr., who was freed last year after serving 15 years in prison for the brutal rape of his then 5-year-old daughter, will be sent back to prison to finish his sentence, the state Supreme Court decided Wednesday. Lavigne was released from prison last year after now-retired Putnam Circuit Judge O.C. Spaulding found there was insufficient evidence to convict him. Lavigne was released on bond following that decision.Putnam County Prosecuting Attorney Mark Sorsaia appealed the decision, saying there was plenty of evidence to convict Lavigne.The Supreme Court unanimously reversed Spaulding's decision, and justices said "we refuse Mr. Lavigne's request to remand the matter back to circuit court; and we order Mr. Lavigne immediately remanded into custody to serve the remainder of his sentence."Lavigne, 54, had served 15 years in prison before being released last spring. In 1996, a Harrison County jury found Lavigne guilty of sexual assault and incest in the brutal rape. The trial was moved from Putnam County because of pretrial publicity. Lavigne was sentenced to 22 to 60 years in prison. Lavigne learned about the decision while he was at work. His attorney Greg Ayers, a Kanawha County public defender, called him at about 3 p.m. to tell him of the decision."I'm not happy. I'm innocent of this crime ... that's not justice, it's not justice," Lavigne said. In a telephone interview Wednesday, he said, "My first parole date is 2021, my discharge date is 2035."He said he planned to appeal the decision to the U.S. Supreme Court.In their opinion, justices said that Spaulding "abused [his] discretion in granting Lavigne a new trial ... without the proper demonstration of constitutional error."In granting Lavigne's release, Spaulding said the father of three had been denied a fair trial because the victim did not identify her father as her attacker in court. He also found that a jury instruction was improper, and limiting Lavigne to four character witnesses fatally harmed his case.During the original trial, according to transcripts, when Lavigne's daughter was asked if she knew her attacker, the court reporter wrote that she looked to her father."...the circuit court went to great pains to discredit the 'supposed glance' towards Mr. Lavigne," the opinion states. Also, justices wrote, "Not only did the circuit court castigate the court reporter for including this parenthetical notation, the circuit court theorized that the glance was fabricated."The opinion states that the number of character witnesses didn't affect the outcome of the case. During the trial, Lavigne's attorney had subpoenaed 12 character witnesses to testify on his behalf, but only four were permitted.The number of character witnesses allowed in a criminal case is up to the trial judge in each case, and justices stood firm on that in their opinion. Then-Circuit Judge Clarence Watt presided over Lavigne's original trial.Justices wrote that the testimony from police, emergency personnel, doctors, and even Lavigne and his then-wife all pointed to Lavigne "as the attacker."Ayers said he believes in Lavigne's innocence and will study the ruling to figure out the next step."I've gotten to know Joe and his family and I believe in his innocence 100 percent," Ayers said. In recent years, Lavigne's daughter, Katie Haught -- now in her 20s with a daughter of her own -- has publicly said her father was not her attacker.It was not clear Wednesday afternoon when exactly Lavigne might return to prison. Sorsaia could not be reached for comment.Lavigne said Wednesday that during his time out of prison, he's enjoyed reconnecting with his children. He planned to cook Thanksgiving dinner and spend as much time as possible with his family before he returns to prison."I've had a year and a half of freedom -- I've rebuilt almost everything I needed to rebuild. I've got a good job, I'm working on getting a good credit rating and as soon as I got that I was going to buy a house," he said. "I was really, really getting my life back together. Now, I'm going to walk through a prison door again."Reach Kate White at email@example.com or 304-348-1723.