Background refresh / indicator #647

Open
opened 11 months ago by prologic · 20 comments
Owner

As a user of the Web Frontend I'd like to be notified of any new updates to my Timeline without having to consstantly refresh the page.

As a user of the Web Frontend I'd like to be notified of any new updates to my Timeline without having to consstantly refresh the page.
prologic added the
enhancement
area/ui
labels 11 months ago
prologic added this to the 0.14 milestone 11 months ago
Owner

Is this something that I'm able to get JS to do with an ajax request? Just have to let me know what endpoint I'd be looking at accessing.

Should be something easy enough to do for 0.14 release?

Is this something that I'm able to get JS to do with an ajax request? Just have to let me know what endpoint I'd be looking at accessing. Should be something *easy* enough to do for 0.14 release?
Poster
Owner

Is this something that I'm able to get JS to do with an ajax request? Just have to let me know what endpoint I'd be looking at accessing.

Should be something easy enough to do for 0.14 release?

This is easy to do from JS yes.

The hard part is auth to the backend.

  • I'm not sure if you can reuse the SESSION cookie to hit a endpoint?
  • If you can't reuse the SESSION cookie, then you'll have to:
  1. Create an endpoint under /api/v1/...
  2. Write a basic client for Yarn's API See archived repos twt.js and twt.js example -- If we revive this library, let's rename it, tidy it up, etc.
  3. Auth to the backend API using the normal /api/v1/auth
  4. Grab the token and shove it in Local Storage for the lgoged in User

Then you can periodically make any AJAX / Fetch API calls you want to the Backend and do interesting things...

> Is this something that I'm able to get JS to do with an ajax request? Just have to let me know what endpoint I'd be looking at accessing. > > Should be something *easy* enough to do for 0.14 release? This is easy to do from JS yes. The hard part is auth to the backend. - I'm not sure if you can reuse the SESSION cookie to hit a endpoint? - If you can't reuse the SESSION cookie, then you'll have to: 1. Create an endpoint under `/api/v1/...` 2. Write a basic client for Yarn's API See archived repos [twt.js](https://git.mills.io/yarnsocial/twt.js) and [twt.js example](https://git.mills.io/yarnsocial/twt.js-example) -- If we revive this library, let's rename it, tidy it up, etc. 3. Auth to the backend API using the normal `/api/v1/auth` 4. Grab the token and shove it in Local Storage for the lgoged in User Then you can periodically make any AJAX / Fetch API calls you want to the Backend and do interesting things...
Owner

I've reviewed the twt.js project and it will need a little bit of work to use it with Yarn.social.

I'm going to earmark this for a 0.16 release as it's more of an enhancement.

I've reviewed the twt.js project and it will need a little bit of work to use it with Yarn.social. I'm going to earmark this for a 0.16 release as it's more of an enhancement.
ullarah modified the milestone from 0.14 to 0.16 8 months ago
Poster
Owner

@markwylde Is this something you can help us build? 🤔

@markwylde Is this something you can help us build? 🤔

@prologic I'm not sure but you can use the credetials: 'include' option in fetch API to send you cookies, I never tried though only used standard token auth with it.

The approach I would go for is something like a simple setInterval with a custom time and a trigger to call the post message.

How is it organised on the page?

@prologic I'm not sure but you can use the `credetials: 'include'` option in fetch API to send you cookies, I never tried though only used standard token auth with it. The approach I would go for is something like a simple `setInterval` with a custom time and a trigger to call the post message. How is it organised on the page?
Poster
Owner

How is it organised on the page?

What do you mean by this? 🤔

> How is it organised on the page? What do you mean by this? 🤔
Poster
Owner

Also what kind of endpointdo do you need to hit, what data do we need to give it and what do you want back? 🤔

Also what kind of endpointdo do you need to hit, what data do we need to give it and what do you want back? 🤔
Owner

This is why I love pure JS frontends and API only backends.

It makes features like this super easy, as you're already using the API to fetch data for your site.

In an API world, I feel you could hit the /api/v1/twts?fromTwt=123456.

That would only return results since the last twt. Where 123456 is a unique id or timestamp. I think twitter uses the timestamp.

Anyway, is a simple dirty solution this:

const parseLatestTwts = html => html.match(/of\ (.*)\ Twts/)[1];

const currentTwts = parseLatestTwts(
  document.querySelector('.timeline-nav').textContent
)

const latestHtmlRequest = await fetch('/')
const latestHtml = await latestHtmlRequest.text()
const latestTwts = parseLatestTwts(latestHtml);

console.log({ currentTwts, latestTwts });

Or just get the first article on the page. But would only tell you if the last article has changed, not how many additional there are.

const parseLatestArticleId = html => html.match(/<article id="(.*)" /)[1];

const currentFirstArticle = parseLatestArticleId(
  document.documentElement.innerHTML
)

const latestHtmlRequest = await fetch('/')
const latestHtml = await latestHtmlRequest.text()
const latestFirstArticle = parseLatestArticleId(latestHtml);

console.log({ currentFirstArticle, latestFirstArticle });
This is why I love pure JS frontends and API only backends. It makes features like this super easy, as you're already using the API to fetch data for your site. In an API world, I feel you could hit the /api/v1/twts?fromTwt=123456. That would only return results since the last twt. Where `123456` is a unique id or timestamp. I think twitter uses the timestamp. Anyway, is a simple dirty solution this: ```javascript const parseLatestTwts = html => html.match(/of\ (.*)\ Twts/)[1]; const currentTwts = parseLatestTwts( document.querySelector('.timeline-nav').textContent ) const latestHtmlRequest = await fetch('/') const latestHtml = await latestHtmlRequest.text() const latestTwts = parseLatestTwts(latestHtml); console.log({ currentTwts, latestTwts }); ``` Or just get the first article on the page. But would only tell you if the last article has changed, not how many additional there are. ```javascript const parseLatestArticleId = html => html.match(/<article id="(.*)" /)[1]; const currentFirstArticle = parseLatestArticleId( document.documentElement.innerHTML ) const latestHtmlRequest = await fetch('/') const latestHtml = await latestHtmlRequest.text() const latestFirstArticle = parseLatestArticleId(latestHtml); console.log({ currentFirstArticle, latestFirstArticle }); ```

@prologic I'm asking how you intend to implement the feature in the code.

Personally I'd recommend a web component (like this one) to build it easily, it can potentially only care for the event and then manage it with a simple event listener to display the banner outside of it.

For the endpoint i'm not really sure but I already did something similar on a semi-live feed at work:

From a Rest API list of data i store the latest ID (from a sequential array) and keep it saved, once the list is refresh (in that case it the visible list and not in background but it doesn't matter much) and check how meny item are ahead of the stored ID, then show a notification and store the new latest ID then repeat the process.

I guess we could go for a similar approach here, or even just a custom that return the latest ID or timestamp in it to check, if we want to handle it as a dedicated API then it could be like this:

  1. get the latest ID on a page (from API) or passed in the html (if with web component)
  2. Store the latest reference in browser storage (Local or session storage)
  3. Call an API with that reference and return a number of new posts
  4. wait with a time and call again
  5. When the page is refreshed everything starts again

That's how it would go for me, we can also do it with the current Post lists API, but tell me if anything is unclear with the steps, maybe i'm missing something.

@prologic I'm asking how you intend to implement the feature in the code. Personally I'd recommend a web component (like [this one](https://git.mills.io/justamoment/yarn-conv-widget)) to build it easily, it can potentially only care for the event and then manage it with a simple event listener to display the banner outside of it. For the endpoint i'm not really sure but I already did something similar on a semi-live feed at work: From a Rest API list of data i store the latest ID (from a sequential array) and keep it saved, once the list is refresh (in that case it the visible list and not in background but it doesn't matter much) and check how meny item are ahead of the stored ID, then show a notification and store the new latest ID then repeat the process. I guess we could go for a similar approach here, or even just a custom that return the latest ID or timestamp in it to check, if we want to handle it as a dedicated API then it could be like this: 1. get the latest ID on a page (from API) or passed in the html (if with web component) 2. Store the latest reference in browser storage (Local or session storage) 3. Call an API with that reference and return a number of new posts 4. wait with a time and call again 5. When the page is refreshed everything starts again That's how it would go for me, we can also do it with the current Post lists API, but tell me if anything is unclear with the steps, maybe i'm missing something.

@markwylde That's right.

Relying on an API is the best choice, even for server side built html, it can be powerful to share the data layer and just render it instead sending it as pure JSON, it would help both the html and any API powered client by staying up to date.

Your solution is really dirty, parsing the html is quite bad.

But I forgot about the total posts count in the response, we could use it without any new endpoint.

The only issue i could see is when the pod decide to clean off the older posts so the number would be on a minus, we should remeber that behavior if we're implementing it like that.

@markwylde That's right. Relying on an API is the best choice, even for server side built html, it can be powerful to share the data layer and just render it instead sending it as pure JSON, it would help both the html and any API powered client by staying up to date. Your solution is really dirty, parsing the html is quite bad. But I forgot about the total posts count in the response, we could use it without any new endpoint. The only issue i could see is when the pod decide to clean off the older posts so the number would be on a minus, we should remeber that behavior if we're implementing it like that.

So, what's the best approach for you?

I can try with my own solution in the meantime.

So, what's the best approach for you? I can try with my own solution in the meantime.
Poster
Owner

@justamoment Are we okay with hitting an endpoint using the existing session/cookie credentials as you mentioend above? If so just need to build something that takes as input the last twt hash on the page and compared it with what's in your timeline in the cache and returns an int? 🤔

@justamoment Are we okay with hitting an endpoint using the existing session/cookie credentials as you mentioend above? If so just need to build something that takes as input the last twt hash on the page and compared it with what's in your timeline in the cache and returns an int? 🤔

@prologic yes for the endpoint, I'm not sure with the cookies, I just read about it but never tried, I can put something together later then we'll see what happens.

Also if we're using this within the page how do we know the latest post of in a different page other than the timeline?

What about the discovery view, conversion and pagination, are they not related to this?

@prologic yes for the endpoint, I'm not sure with the cookies, I just read about it but never tried, I can put something together later then we'll see what happens. Also if we're using this within the page how do we know the latest post of in a different page other than the timeline? What about the discovery view, conversion and pagination, are they not related to this?
Poster
Owner

What if we do something like this:

  1. On a timer do a fetch() call with a query string like ?countonly&from=<hash>
  2. The same view on the backend prepares what it would normally but sees the query string and just returns a JSON object with {"count": N}.
  3. THen display the banner with "There are N new twts, Click here to refresh"

Simple enough? 🤔

What if we do something like this: 1. On a timer do a `fetch()` call with a query string like `?countonly&from=<hash>` 2. The same view on the backend prepares what it would normally but sees the query string and just returns a JSON object with `{"count": N}`. 3. THen display the banner with "There are N new twts, Click here to refresh" Simple enough? 🤔

@prologic looks good to me.

If we add in the response the hash of the latest post too we can also handle it cumulatively, like:

  • 1 new twt
  • 2 new twts
  • 1 new twt

for a total of 4 twts, does it make sense?

@prologic looks good to me. If we add in the response the hash of the latest post too we can also handle it cumulatively, like: - 1 new twt - 2 new twts - 1 new twt for a total of 4 twts, does it make sense?

@prologic This way we could check only for newer post after the first notification came out.

@prologic This way we could check only for newer post after the first notification came out.
Poster
Owner

@justamoment I think makes sense. I'll see if I can create such an endpoint and roll it out to my pod. How do you prefer to do dev on the UI/UX side here? A local dev pod or code directly against my pod live? 🤔

@justamoment I _think_ makes sense. I'll see if I can create such an endpoint and roll it out to my pod. How do you prefer to do dev on the UI/UX side here? A local dev pod or code directly against my pod live? 🤔

@prologic I think I can put up a local instance to dev on, I'm not sure how to work on the JS though.

But if you're okay with a web-component I can make it even directly in the browser if the API is accessible.

@prologic I think I can put up a local instance to dev on, I'm not sure how to work on the JS though. But if you're okay with a web-component I can make it even directly in the browser if the API is accessible.
Poster
Owner

@prologic I think I can put up a local instance to dev on, I'm not sure how to work on the JS though.

We don't do anything "fancy". Just shove *.js files in ./internal/theme/static/js/ (no package manageer) and name them appropriately (if they need to be defined in a certain order) and just write simple shit™ 😅

> @prologic I think I can put up a local instance to dev on, I'm not sure how to work on the JS though. We don't do anything "fancy". Just shove `*.js` files in `./internal/theme/static/js/` (no package manageer) and name them appropriately (if they need to be defined in a certain order) and just write simple shit™ 😅

@prologic Great! 👍

I'll be as simple and reusable as i can. 😎

@prologic Great! 👍 I'll be as simple *and* reusable as i can. 😎
Sign in to join this conversation.
No Milestone
No Assignees
4 Participants
Notifications
Due Date

No due date set.

Dependencies

No dependencies set.

Reference: yarnsocial/yarn#647
Loading…
There is no content yet.