PEP 484 introduced type hints, at this time documenting exceptions was left to docstrings. I seek to suggest a reason this feature might be desirable along with how it might be used. Error handling in python does an excellent job of keeping the error-path out of the way when writing the normal flow of logic, however for larger code bases it is not always clear what exceptions may be caused by calling existing code. Since these cases are easily missed they may reach a higher level than intended ...
This is a discussion on Python’s forums about adding something akin to a throws keyword in python.
Handling can mean a lot of things. You can use a sigil to quickly return early from the function without cluttering up your code. For example, in Rust (code somewhat invalid because I couldn’t post the generic arg to Result because lemmy formatting rules):
fn my_func() -> Result {
let val = some_func_that_can_error()?;
return Some(val.operation_that_can_error());
}
let val = match my_func() {
Err(err) => {
println!("Your error: {err}");
return;
}
Some(val) => val,
};
// use val here
That question mark inside my_func shows the programmer that there’s a potential error, but that the caller will handle it.
I’m suggesting something similar for Python, where you can easily show that there’s a potential error in the code, without having to do much to deal with it when it happens if the only thing you want to do is bubble it up.
If we use exceptions, it isn’t obvious where the errors could occur, and it’s easy to defer handling it much too late unless you want to clutter your code.
@sugar_in_your_tea I’m by far not qualified to discuss this in depth. But it seems to me that almost every function call ever can fail. Therefore, do you need to do this with every single function call?
That seems terribly inefficient and bloated. How is that readable for anyone?
That’s where the difference between exceptional cases comes in. Rust and Go both have the concept of a panic, which is an error that can only be caught with a special mechanism (not a try/except).
So that’ll cover unexpected errors like divide by zero, out of memory, etc, and you’d handle other errors as data (e.g. record not found, validation error, etc).
I don’t think Python should necessarily go as far as Go or Rust, just that handling errors like data should be an option instead of being forced to use try/except, which I find to be gross. In general, I want to use try/except if I want a stack trace, and error values when I don’t.
@sugar_in_your_tea But isn’t all that possible in Python? Don’t monads cover exactly what you want? Why does it need to be implemented some different way?
Also, divide by zero should be data just as well. Failing to program around having nothing to divide by is not a reason to have a program panic.
Also, having two systems for largely the same behavior doesn’t seem to improve usability and clarity, in my opinion.
I disagree. You should be checking your input data so the divide by zero is impossible. An invalid input error is data and it can probably be recovered from, whereas a divide by zero is something your program should never do.
If having the error is expected behavior (e.g. records/files can not exist, user data can be invalid, external service is down, etc), it’s data. If it’s a surprise, it’s an exception and should crash.
doesn’t seem to improve usability
I’m proposing that the programmer chooses. The whole design ethos around Python is that it should look like pseudocode. Pseudocode generally ignores errors, but if it doesn’t, it’s reasonable to express it as either an exception or data.
Documenting functions with “throws” isn’t something I’d do in pseudocode because enumerating the ways something can fail generally isn’t interesting. However, knowing that a function call can fail is interesting, so I think error passing in the Rust way is an interesting, subtle way of doing that.
I’m not saying we should absolutely go with monadic error returns, I’m saying that if we change error handling, I’d prefer to go that route than Java’s throws, because I think documenting exceptions encourages bad use of exceptions. The code I work on already has way too many try/except blocks, I’m concerned this would cement that practice.
@sugar_in_your_tea Since when is Python supposed to equal pseudo code? It should be easily readable, but that doesn’t mean it should *equal* pseudo code.
You can either test for values being 0 before dividing, or catching an exception when it is. Especially when dividing multiple times in one function, I would go for the latter option.
It’s not an explicit design goal, but it explains a lot of the Zen of Python and other pushback on PIPs, so to me it’s always been an unwritten design goal (be as close to pseudocode as practical, but no closer). It’s also how I generally write code (start with Python “pseudocode,” then decide what to use in production).
For example, from the Zen of Python:
There should be one-- and preferably only one --obvious way to do it.
Being clever in Python is a bad thing, just as it is in pseudocode. Python will never win awards for performance, so if you need that, you drop in something non-Python to do the expensive operations to keep the rest of the code clean and obvious.
If you think of Python as pseudocode, everything else makes a ton more sense.
You can test for values being 0 before dividing, or catching an exception when it is.
Ideally, you just test for input variables outside of the function and do neither. Something like:
defcalc(x, y):
assert x > 0assert y != 0
...
This throw exceptions if the preconditions fail, but those can (and should) be removed for production since their primary purpose is to inform the developer of the preconditions and catch mistakes in development. In production, you’d rely on some kind of schema validation to ensure the asserts never trigger (I’m partial to Pydantic).
So ideally you’d never expect a divide by zero or clutter your code with zero checks outside of those asserts (which shouldn’t be relied on) because you’ve already prevented those cases from happening.
@sugar_in_your_tea Using asserts in any code except testing is frowned upon, afaik. You should use specific exceptions instead of vague unlabeled assertion errors.
You also seem to think that you’re not allowed to use exception to communicate the fact a check failed. If that’s the case, you’re seriously underusing the power of exceptions.
It sounds a lot to me like you don’t even want to use Python or think it shouldn’t be used for anything serious. Why then even argue about it?
Assertion errors should never fire, they’re merely there for documentation and catching mistakes in development. Any assertion is merely a sanity check (the value should’ve been checked before calling the function), which is why they’re disabled in production.
In fact, I conceptually like the way D makes checking preconditions and postconditions explicit. However, it’s clunky in practice imo, so asserts are usually elegant enough. I honestly only use asserts when it’s the clearest way to document the usage constraints.
you’re seriously underusing the power of exceptions
No, I use them for communicating data errors and whatnot and have a bunch of custom exceptions in my code. It’s the current Pythonic way, so that’s what I do.
However, I don’t like that pattern and find it to either hide errors or clutter my code. I much prefer the Rust style of error handling where errors are always acknowledged when they can happen, but usually handled at a higher level (like you’d do in Python, but with explicit syntax to acknowledge a call could error). I find this gives me, the programmer, a chance to consider the error case to correct logical mistakes before actually running any code, and it also improves code reviews because it’s obvious to the reviewer that the code could error. I’ve had far too many bugs caused by not knowing or forgetting a call could raise an error.
you don’t even want to use Python
When did I say that? I use it at my day job and actually argued against using Rust for our project because Python maps really well to our problem domain. Our project is hundreds of thousands of lines of Python across a dozen or more microservices, and it has served us well.
Criticizing a language doesn’t necessarily mean I don’t like it, it just means I think it could be better. Python is generally my first choice unless I know I need top performance and correctness out of the gate. For example, I’m writing a distributed lemmy competitor in Rust in my free time for various reasons (mostly I don’t want to deal with Python installers, and there’s no server component), and also building a game in Godot in GDScript (very similar to Python, even worse in perf). There are very few languages I actively dislike.
That said, in general, I prefer functional-style programming, and exceptions are one glaring wart that makes FP in Python feel bad. I want that to be better.
@sugar_in_your_tea I don’t think we should change any functionality when it comes to exception handling. Code based documentation would be great for type checking and auto-generated docs, but they can be done using annotations, not changed interfaces.
Monads are already possible, but should not be the normal way to code either. It’s clunky and difficult to understand. It might work great for some scenarios, but doesn’t for many others.
Monads are only clunky because Python doesn’t really support them.
And I agree, I don’t think we should change existing exception handling, just allow the programmer to interact with it differently. I’d love to be able to turn exceptions into monads with a little bit of syntax. Under the hood, Python would still do the try/except, but my code would use exceptions as values instead. You’d still be able to use the older try/except explicitly just like you can express a list comprehension as a generator manually, you’d just have the option to do something else if it’s cleaner in your project.
That said, exceptions as values isn’t a hill I’m willing to die on, but I will push against “throws” being added, and optional chaining is a hill I’m willing to die on.
Handling can mean a lot of things. You can use a sigil to quickly return early from the function without cluttering up your code. For example, in Rust (code somewhat invalid because I couldn’t post the generic arg to Result because lemmy formatting rules):
fn my_func() -> Result { let val = some_func_that_can_error()?; return Some(val.operation_that_can_error()); } let val = match my_func() { Err(err) => { println!("Your error: {err}"); return; } Some(val) => val, }; // use val hereThat question mark inside
my_funcshows the programmer that there’s a potential error, but that the caller will handle it.I’m suggesting something similar for Python, where you can easily show that there’s a potential error in the code, without having to do much to deal with it when it happens if the only thing you want to do is bubble it up.
If we use exceptions, it isn’t obvious where the errors could occur, and it’s easy to defer handling it much too late unless you want to clutter your code.
@sugar_in_your_tea I’m by far not qualified to discuss this in depth. But it seems to me that almost every function call ever can fail. Therefore, do you need to do this with every single function call?
That seems terribly inefficient and bloated. How is that readable for anyone?
That’s where the difference between exceptional cases comes in. Rust and Go both have the concept of a panic, which is an error that can only be caught with a special mechanism (not a try/except).
So that’ll cover unexpected errors like divide by zero, out of memory, etc, and you’d handle other errors as data (e.g. record not found, validation error, etc).
I don’t think Python should necessarily go as far as Go or Rust, just that handling errors like data should be an option instead of being forced to use try/except, which I find to be gross. In general, I want to use try/except if I want a stack trace, and error values when I don’t.
@sugar_in_your_tea But isn’t all that possible in Python? Don’t monads cover exactly what you want? Why does it need to be implemented some different way?
Also, divide by zero should be data just as well. Failing to program around having nothing to divide by is not a reason to have a program panic.
Also, having two systems for largely the same behavior doesn’t seem to improve usability and clarity, in my opinion.
I disagree. You should be checking your input data so the divide by zero is impossible. An invalid input error is data and it can probably be recovered from, whereas a divide by zero is something your program should never do.
If having the error is expected behavior (e.g. records/files can not exist, user data can be invalid, external service is down, etc), it’s data. If it’s a surprise, it’s an exception and should crash.
I’m proposing that the programmer chooses. The whole design ethos around Python is that it should look like pseudocode. Pseudocode generally ignores errors, but if it doesn’t, it’s reasonable to express it as either an exception or data.
Documenting functions with “throws” isn’t something I’d do in pseudocode because enumerating the ways something can fail generally isn’t interesting. However, knowing that a function call can fail is interesting, so I think error passing in the Rust way is an interesting, subtle way of doing that.
I’m not saying we should absolutely go with monadic error returns, I’m saying that if we change error handling, I’d prefer to go that route than Java’s throws, because I think documenting exceptions encourages bad use of exceptions. The code I work on already has way too many try/except blocks, I’m concerned this would cement that practice.
@sugar_in_your_tea Since when is Python supposed to equal pseudo code? It should be easily readable, but that doesn’t mean it should *equal* pseudo code.
You can either test for values being 0 before dividing, or catching an exception when it is. Especially when dividing multiple times in one function, I would go for the latter option.
It’s not an explicit design goal, but it explains a lot of the Zen of Python and other pushback on PIPs, so to me it’s always been an unwritten design goal (be as close to pseudocode as practical, but no closer). It’s also how I generally write code (start with Python “pseudocode,” then decide what to use in production).
For example, from the Zen of Python:
Being clever in Python is a bad thing, just as it is in pseudocode. Python will never win awards for performance, so if you need that, you drop in something non-Python to do the expensive operations to keep the rest of the code clean and obvious.
If you think of Python as pseudocode, everything else makes a ton more sense.
Ideally, you just test for input variables outside of the function and do neither. Something like:
def calc(x, y): assert x > 0 assert y != 0 ...This throw exceptions if the preconditions fail, but those can (and should) be removed for production since their primary purpose is to inform the developer of the preconditions and catch mistakes in development. In production, you’d rely on some kind of schema validation to ensure the asserts never trigger (I’m partial to Pydantic).
So ideally you’d never expect a divide by zero or clutter your code with zero checks outside of those asserts (which shouldn’t be relied on) because you’ve already prevented those cases from happening.
@sugar_in_your_tea Using asserts in any code except testing is frowned upon, afaik. You should use specific exceptions instead of vague unlabeled assertion errors.
You also seem to think that you’re not allowed to use exception to communicate the fact a check failed. If that’s the case, you’re seriously underusing the power of exceptions.
It sounds a lot to me like you don’t even want to use Python or think it shouldn’t be used for anything serious. Why then even argue about it?
Assertion errors should never fire, they’re merely there for documentation and catching mistakes in development. Any assertion is merely a sanity check (the value should’ve been checked before calling the function), which is why they’re disabled in production.
In fact, I conceptually like the way D makes checking preconditions and postconditions explicit. However, it’s clunky in practice imo, so asserts are usually elegant enough. I honestly only use asserts when it’s the clearest way to document the usage constraints.
No, I use them for communicating data errors and whatnot and have a bunch of custom exceptions in my code. It’s the current Pythonic way, so that’s what I do.
However, I don’t like that pattern and find it to either hide errors or clutter my code. I much prefer the Rust style of error handling where errors are always acknowledged when they can happen, but usually handled at a higher level (like you’d do in Python, but with explicit syntax to acknowledge a call could error). I find this gives me, the programmer, a chance to consider the error case to correct logical mistakes before actually running any code, and it also improves code reviews because it’s obvious to the reviewer that the code could error. I’ve had far too many bugs caused by not knowing or forgetting a call could raise an error.
When did I say that? I use it at my day job and actually argued against using Rust for our project because Python maps really well to our problem domain. Our project is hundreds of thousands of lines of Python across a dozen or more microservices, and it has served us well.
Criticizing a language doesn’t necessarily mean I don’t like it, it just means I think it could be better. Python is generally my first choice unless I know I need top performance and correctness out of the gate. For example, I’m writing a distributed lemmy competitor in Rust in my free time for various reasons (mostly I don’t want to deal with Python installers, and there’s no server component), and also building a game in Godot in GDScript (very similar to Python, even worse in perf). There are very few languages I actively dislike.
That said, in general, I prefer functional-style programming, and exceptions are one glaring wart that makes FP in Python feel bad. I want that to be better.
@sugar_in_your_tea I don’t think we should change any functionality when it comes to exception handling. Code based documentation would be great for type checking and auto-generated docs, but they can be done using annotations, not changed interfaces.
Monads are already possible, but should not be the normal way to code either. It’s clunky and difficult to understand. It might work great for some scenarios, but doesn’t for many others.
Monads are only clunky because Python doesn’t really support them.
And I agree, I don’t think we should change existing exception handling, just allow the programmer to interact with it differently. I’d love to be able to turn exceptions into monads with a little bit of syntax. Under the hood, Python would still do the try/except, but my code would use exceptions as values instead. You’d still be able to use the older try/except explicitly just like you can express a list comprehension as a generator manually, you’d just have the option to do something else if it’s cleaner in your project.
That said, exceptions as values isn’t a hill I’m willing to die on, but I will push against “throws” being added, and optional chaining is a hill I’m willing to die on.