Problem: 

The customer has enabled 3 decimal places in culture information. 

And the transaction total is 5.462

Now the customer wants the Transaction total should be 5.460 on payment screen and on payment of 100 change amount should be 94.54

Reason :  

Changing the Rounding method to RoundtoFivethousandth in Payment Type fixed this problem.