If 5, 5r, 5r^2 are the lengths of the sides of a triangle, then r cannot be equal to:

5, 5r, 5r^2 sides of triangle,
5 + 5r > 5r^2.. (1)
5 + 5r^2 > 5r.. (2)
5r + 5r^2 > 5.. (3)
From (1)
r^2 - r - 1 < 0,
[r - (1 + √5/2)][r - (1 - √5/2)] < 0
r ∈ (1 - √5/2, 1 + √5/2).. (4)
from (2), r^2 - r + 1 > 0 ⇒ r ∈ R.. (5)
from (3), r^2 + r - 1 > 0
So, (r + 1 + √5/2)(r + 1 - √5/2) > 0
r ∈ (-∞, -1 - √5/2) ∪ (-1 + √5/2, ∞).. (6)
from (4), (5), (6), r ∈ (-1 + √5/2, 1 + √5/2)