<ocdtrekkie>
Ian, can you turn on issues on your MediaWiki repo?
<ocdtrekkie>
It's mostly serviceable (I played with it an hour before I found an issue) but now I found two.
<isd>
Done. I guess it's off for forks by default?
koo5_ has joined #sandstorm
<ocdtrekkie>
Yeah it is
<ocdtrekkie>
Which makes sense if you're forking just to contribute upstream.
<ocdtrekkie>
I have an old wiki on shared hosting last edited in like 2012. I am migrating it into this page by page so I can take it down without losing the ability to access it or share it if someone wants access to it.
<ocdtrekkie>
The fun part about testing the app this way is I'm getting real production data, so... edge cases!
<ocdtrekkie>
@kentonv Did you see the Meteor 2.7.2 stuff?
<kentonv>
ocdtrekkie, the fix for the index conflict issue, or something else?
<ocdtrekkie>
That, though you have to enable it
mnutt has quit [Remote host closed the connection]
<kentonv>
Hmm, how do I do that?
<kentonv>
Meteor.settings.packages.mongo.reCreateIndexOnOptionMismatch I guess
<ocdtrekkie>
yeah, was digging it up
<ocdtrekkie>
So in theory if Sandstorm sets that, it should be possible to upgrade Alpha and all.
mnutt has joined #sandstorm
<kentonv>
this means we're updating 2.3.5 -> 2.7.2... I wonder how much other stuff broke in the meantime. -_-
mnutt has quit [Ping timeout: 240 seconds]
<ocdtrekkie>
Hmmm, maybe not a great project for a narrow time window.
<kentonv>
hmm, Meteor 2.6 advertises the oldest supported Mongo version as being 3.6, but of course we're stuck on 2.6...
<kentonv>
lolz: MongoServerSelectionError: Server at 127.0.0.1:9001 reports maximum wire version 2, but this version of the Node.js Driver requires at least 6 (MongoDB 3.6)
<ocdtrekkie>
So the Mongo upgrade problem isn't really avoidable anymore.
<isd>
Yeah, we need to figure out something.
<kentonv>
and Mongo's built-in upgrade path requires you to be on the latest version of the previous major release before you can update to the next. So if we use that, we have to ship 4 versions of mongo and apply three separate upgrade steps.
<kentonv>
probably easier to dump everything to JSON or some such and then import it again
<isd>
Yeah, probably.
<kentonv>
that still requires us to ship the old Mongo binary forever, of course
<kentonv>
hopefully the script we need to write can run inside Mongo's embedded JS environment so we don't need to ship a separate stack with an old driver forever.
<isd>
One alternative to keeping the binary around forever would be to have the upgrade logic know enough to use intermediate versions if the version upgrading is too old. I think some of that is actually in place, at least the bit where sandstorm reports its own version when asking for an update.
<isd>
So the version that dropped the binary would require you to have a version whose database is already migrated, and the update server would direct you to the prior version if it isn't.
<isd>
...but, that's longer term. We can ship the binary for a while at least.
<kentonv>
yeah it's... not *that* bad keeping the binary, if that's all we reed
<kentonv>
*need
<ocdtrekkie>
I'm rapidly starting to believe all databases should probably just be SQLite. :P
<kentonv>
it would definitely have been the right choice for self-hosted sandstorm
<isd>
Mongo uses some format that's supports dates and a couple other not-json things right? So we probably can't just do JSON.stringify({grains: db.grains.find({}), packages: db.packages.find({}) ...
<isd>
...also I kindof expect that to choke on a large db.
<kentonv>
yeah. We also have binary blobs in some places I think.
<isd>
we probably can't just run it in the mongo shell then...
<kentonv>
on the bright side the test suite is down to only one failure for me
<kentonv>
tests/account
<isd>
Yeah, I think two of them are just flaky
<isd>
Is that the one that tries to do stuff with demo accounts?
<isd>
...maybe we can gradually migrate away from mongo, a table/collection at a time.
<isd>
(It's not like it has joins)
<ocdtrekkie>
Does Meteor support everything with SQLite as a database backend now?
<isd>
I do not think so.
<ocdtrekkie>
Or some other database?
<ocdtrekkie>
Because I know in 2014 Meteor only worked with Mongo, but I thought they expanded past that.
<isd>
(If we ever go to the trouble of moving to something else, it should be something whose storage format is extremely stable, like sqlite)
mnutt has joined #sandstorm
mnutt has quit [Remote host closed the connection]
mnutt has joined #sandstorm
mnutt has quit [Ping timeout: 248 seconds]
mnutt has joined #sandstorm
xet7 has quit [Remote host closed the connection]
xet7 has joined #sandstorm
mnutt has quit [Remote host closed the connection]
<TimMc>
ocdtrekkie: sqlite 4ever
<ocdtrekkie>
kentonv: Did you move the website? :O
<ocdtrekkie>
That looked magical.
<kentonv>
So I just migrated sandstorm.io to be hosted on Cloudflare Pages. It took... 40 minutes. And that includes time to write a worker that falls back to the old server for the few files that aren't in the repo (old spks, mainly), including logging the filenames that are actually accessed so we can move them to R2 later.
<kentonv>
the web site build itself worked 100% out of the box, except that we didn't have a 404 page so I had to add that.
<ocdtrekkie>
Neat!
<kentonv>
and yes, now every pull request will get a reply from Cloudflare within a minute or so with a link to a preview
<kentonv>
and will automatically deploy upon merge
<ocdtrekkie>
Should I work on a PR to change the corporate speak stuff?
<kentonv>
sure!
<ocdtrekkie>
I found an interesting behavior of Pages here
<kentonv>
?
<ocdtrekkie>
It seems the first time I load a given URL (to test the 404) it reveals the pages domain.
<kentonv>
what do you mean?
<ocdtrekkie>
So if I go to sandstorm.io/neverbeforeseen I get sandstorm-website.pages.dev/neverbeforeseen in the URL bar
<ocdtrekkie>
Nope, I'm wrong, it's an HTTP redirect artifact.
<ocdtrekkie>
My browser is trying HTTP first, and if you visit sandstorm.io URLs without a protocol, defaulting in my case to HTTP, the redirect puts you on the pages.dev address.
<kentonv>
oh interesting
<kentonv>
I can fix that
<kentonv>
that's a bug in my worker, not a bug in pages
<ocdtrekkie>
yay testing \o/
<kentonv>
I didn't map pages onto sandstorm.io in the usual way, which would be by setting up a DNS cname. I didn't want to do that because I still want to send some requests through to the old origin server, when pages returns 404.
<kentonv>
so instead I have a worker that tries to fetch from sandstorm-website.pages.dev first
<kentonv>
fixed
<ocdtrekkie>
woo!
<kentonv>
now I wonder if I can convince pages to do Cap'n Proto's web site, considering that it requires an outdated version of Jekyll.....
<kentonv>
I guess it's a problem that the Cap'n Proto web site is in the same repo as the code... any PR to the code would trigger a Pages build which is not what I want.
<ocdtrekkie>
Can you limit the Pages build to only look at specific folders like GitHub Actions?
<kentonv>
looks like I can make it match only specific branch names, that might be good enough
<ocdtrekkie>
Oh, and is the Mail Channels thing or whatever it was called going to be useful for moving Sandcats over to Workers?
<kentonv>
I don't think it provides any new capabilities beyond what we could do with sendgrid, but it might be better-integrated