It’s a wrapper for stuff written in c, right? So they’re trying to prevent segfaults which are less readable

Not C per se, just sometimes cython.  I was thinking about this sort of thing, but I just don’t buy it as an objection – even if there are places where things get low level, the checks should only come right before those parts (they are much more frequent than that).

And, more importantly, even if the constraint is “we need data that’s exactly like this, at this point,” there are ways for developers to respond to that constraint other than “insist the user of our API satisfy it themselves, and break if they happen not to.”  Like, you can surround your low level code with Python/numpy stuff that intelligently processes it into the required form, to get the behavior expected by Python/numpy users.  This can be frustrating to write, sure – it’s similar to what I’m doing to get around sklearn – which I guess might be why it isn’t there?

But also it’s just less frustrating to see this error the first time than scratch your head if the assumptions are violated in some subtle way

That’s what warnings are for!

eaglesnotforks replied to your post “Does anyone know why scikit-learn is (pervasively) designed to raise…”

I don’t know, but I hate it and it’s made my past month exceedingly frustrating. So…solidarity, I guess.

At this point I’m half-seriously wondering if it would save more time just to make a fork of sklearn that turns all the frustrating exceptions into warnings, rather than spending hours and hours on workarounds.

Of course it wouldn’t get updated (or rather, I wouldn’t want to deal with the headache of merging updates into it and de-exceptionizing them), but I’m not sure anything in a future sklearn update is going to have much value for me?  It already does a whole bunch of more-or-less standard stuff and that’s all I want out of it.

Does anyone know why scikit-learn is (pervasively) designed to raise exceptions whenever anything is a little unusual in a way that could maybe cause problems down the line, even if it’s not causing problems now?

Like, it has all these checks for things like whether data has the right type(s), or the right number or dimensions, or the right range of values in it, or whatever – where “right” usually means “necessary for doing something that is commonly done with this value at some point.”  But instead of applying these checks only when it’s about to do something that requires those properties, it applies them continually, again and again, inside every tiny moving part, even parts that have no reason of their own to care about the properties.  And instead of raising warnings, it raises exceptions, i.e. it just stops and refuses to continue.

This requires some pretty ridiculous workarounds if you want to do a lot of things that would be straightforward in other big Python libraries or just Python itself.  Is there some secret good reason it’s built this way?

I remember only the final minutes of last night’s dream: I was in a car, around lunchtime, with two friends (dream characters), and the friend driving the car casually revealed their plan to drive out to the beach, hunt a whale, bring its carcass (or part of it?) back home, and cook whale meat for lunch.  (Specifically for lunch, even though it was nearly lunchtime – they seemed to think, not just that this could all be done, but that it could all be done very quickly.)  They showed us a crude harpoon they’d made (not sure exactly how it was lodged into the car).

On Mars

tanadrin:

Things I love about Mars: the landscape.

Mars’s landscape is both alien and familiar. There are other fascinating landscapes in the Solar System, of course: Venus, Pluto, Europa, Titan, etc., and each has their charms; but the thing about those landscapes is that the environment in which they’re found makes them more alien. Venus has mountains and plains and, like Earth, few craters; but the crushing sulphuric pressure of the atmosphere and the fact that every few hundred million years it seems the entire planet may go molten and resurface itself makes Venus a setting for hard SF, or individualistic person-versus-environment stories: the narrative that suggests itself to me when I imagine standing (in some megaspacesuit) on the surface of Venus is not “this is a place humans could one day be,” but “this is an unpeopled Hell.”

(Also: apparently Venus may have had liquid water as recently as 700 MYA. Life on Earth seems to have arisen almost immediately, as soon as the conditions potentially favorable to it existed. From the formation of its oceans to 700 MYA, Venus would have been climactically stable, thanks to higher cloud cover than Earth. So it is entirely possible that for a couple of billion years, between the oceans of Venus forming and the runaway greenhouse effect destroying them several hundred MYA, Venus had life, up until the Neoproterozoic period on Earth. But if the theories regarding how energy is released into Venus’s dessicated crust are correct, the fossil evidence of that life would have been annihilated in the same event that resurfaced the entire planet some time in its geolocially recent past. Perhaps fragments of it persist, floating deep in the mantle like the Farallon plate on Earth–but for now, an actual record of the biohistory of Venus is lost to us. What I’m saying is, Venus is a postapocalypse: not a hopeful Perelandra, not even in the far future, but a grievous memorial for what might have been our lush and gardenlike neighbor.)

Keep reading