I have been approached several times by Railroad authorities, local law enforcement officers and the public. As far as the Railroad goes, the demarcation between Railroad property and public property is rarely marked so it is very easy to end up on the wrong piece of grass.
The Railroads own vast amounts of real estate and it can extend some distance from the tracks, but generally speaking if you're in an area where railfans populate then you'll usually not be bothered, however, us Brits like to look for something different (I call it prospecting) location wise so crews will not be used to seeing railfans there and will usually call up.
I have only once been asked to leave and to be fair I was trespassing, although totally unaware I was doing so; I'd been there (in town) for a few days and the employee said I had my fun and perhaps it was time to move on (read the local swing helper crews had had enough of me poking my camera at them). I apologised, thanked him for the explanation on company property, wished him good day and %^&* off to take pictures a 100 miles away on a different Railroad.
I spoke to a railfan about it and he said, ahhh, the trick is to take a couple of photos and then sod off somewhere else, that way they never send the crews out to catch you, it's only if you 'squat' in the same location all day they get suspicious.
Some locals get upset, call the law who duly arrive, have a chat, check your details and depending on how they feel either ask you to move on or stay until the next train and then move on, but that was only once, all the other interactions have been favourable, interesting and polite. Quite a few of my local sheriff interactions were from locals who called up and concern that I was going to jump under.
Given I did over 7,000 miles in three weeks on my last trip, there's plenty of railway to go to and take photos, so I just move on and enjoy the day.
On the whole I prefer areas out of town and away from people, but then I've been approached by ranchers and farm hands and questioned but those have all been amicable, more so if you explain you're only going to be an hour or so and move on; some have even suggested better spots and given directions to them.
I must have 3-4000 US train photos (not including the 10,000 roster and detail shots) and only once have I been given the 'finger' by a disgruntled engineer.