Closed Bug 1196233 Opened 9 years ago Closed 6 years ago

Add support for Physical Web to find devices around you

Categories

(Firefox OS Graveyard :: Gaia::System::Lockscreen, defect)

Product:
Firefox OS Graveyard
Component:
Gaia::System::Lockscreen
Platform:
ARM
Gonk (Firefox OS)
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: janjongboom, Unassigned)

Details

Attachments

(5 files)

[gaia] comoyo:physical_web > mozilla-b2g:master
46 bytes, text/x-github-pull-request
Details | Review
Lockscreen UI example 1
70.46 KB, image/png
Details
Lockscreen UI example 2
157.00 KB, image/png
Details
UX Screen 1 (devices found)
231.71 KB, image/png
Details
UX Screen 2 (more info)
265.47 KB, image/png
Details 
Open for discussion. Physical Web is an open standard initiated by Google to allow devices to broadcast URLs over Bluetooth Low Energy. This allows f.e. a meeting room to broadcast it's agenda; a movie poster to broadcast viewing times; etc. This all happens on the web, no apps needed. 

Could be a great push for the mobile web, and especially with Web Bluetooth coming to Firefox OS this'd be a great experience (discover and fly a drone in 5 seconds w/o installing an app!).

I made a patch, and let's see whether we're interested in taking this upstream...

Video: https://www.youtube.com/watch?v=DynTYekCsJo&feature=youtu.be
@harly could you bring this topic in UX discussion? 
With new Bluetooth LE API we finally can do these great things in our OS.

Note that We've generated several ideas around nearby & physical web scenarios during idea sprint, so we can recheck them if necessary.
forgot to set ni :p
Flags: needinfo?(hhsu)
This is an awesome demo of the physical web, and I can see many useful use cases. However, currently there are no system wise UX designs for physical web, so I would suggest to have this as an add-on or an option in developer settings. Once we have a UX design, we can gradually incorporate physical web into Firefox OS.
Flags: needinfo?(hhsu)
I'm happy to offer design support here if that would help. I'm afraid I don't have any templates/guidelines to properly build an appropriate design. However, if you could point me to anything that would help, plus a few words of what you'd like to see in terms of deliverables, I'm happy to give this some time.
Scott, that would be fantastic! Harly, can you provide the link to the UX team templates so Scott can take a stab at a design for this?
Flags: needinfo?(hhsu)
Hi Scott,
Thanks for providing help on this, and below is a link to MDN to get you started:
https://developer.mozilla.org/en-US/Apps/Design

I would strongly suggest you to start taking a look at Firefox OS Building Blocks and Firefox OS Visual styleguide, it will help you get familiar with our design much quicker.
Flags: needinfo?(hhsu)
Nice to see things going on.

As a heavy tech user, I'd like to propose integrate such feature in search bar instead of in lockscreen, the reasons are:

1. find/reach things from the search bar is more webby
2. not everyone enabled lockscreen. And when user wants to find physical web, they need go back to lockscreen.
3. while user try to search something, they have the power in their hand to search the internet, **scan & filter local physical web**, app, and the app content in device.
4. we can integrate search bar in lockscreen so we can still access physical web lockscreen. (If its a good UX)


FYR, the Firefox OS Building Blocks that harly mentioned https://developer.mozilla.org/en-US/Apps/Design/Firefox_OS_building_blocks
Jan, how about you bundle this up into a standalone web-component so that is can be dropped anywhere? That way we can continue to maintain the component and won't risk this valuable work rotting.
Flags: needinfo?(janjongboom)
wilsonpage: I don't see the benefit at this stage, we don't even know whether the current interaction is the 'right' one according to UX.

It's however pretty much self contained. Almost everything lives in apps/system/lockscreen/js/lockscreen_physical_web.js. Only thing that could rot is UI as we're piggy backing on notification UI. But we have no clue what we want with that anyway.
Flags: needinfo?(janjongboom)
(In reply to Fred Lin [:gasolin] from comment #10)
> Nice to see things going on.
> 
> As a heavy tech user, I'd like to propose integrate such feature in search
> bar instead of in lockscreen, the reasons are:
> 
> 1. find/reach things from the search bar is more webby
> 2. not everyone enabled lockscreen. And when user wants to find physical
> web, they need go back to lockscreen.
> 3. while user try to search something, they have the power in their hand to
> search the internet, **scan & filter local physical web**, app, and the app
> content in device.
> 4. we can integrate search bar in lockscreen so we can still access physical
> web lockscreen. (If its a good UX)

Actually I think in general physical web should be part of the browser (even on Firefox if we're running on a computer with BLE support), and in that sense the search bar in FxOS would be a nice place to put. However it requires a lot more actions from the generic user (unlock, tap search bar, tap pw button, plus reduced visibility) than when we integrate in lock screen as well. I believe physical web could be an important usecase where we can show the benefit of mobile web over native again, and thus think we should make it an important interaction.
(In reply to Jan Jongboom [:janjongboom] (Telenor) from comment #12)
> wilsonpage: I don't see the benefit at this stage, we don't even know
> whether the current interaction is the 'right' one according to UX.
> 
> It's however pretty much self contained. Almost everything lives in
> apps/system/lockscreen/js/lockscreen_physical_web.js. Only thing that could
> rot is UI as we're piggy backing on notification UI. But we have no clue
> what we want with that anyway.

Fair enough. I'm imagining that we would want discovery in several places across the OS (eg. lockscreen & utility-tray), so a portable component would be valuable here.

I vote we land this behind a developer-menu pref and let foxfooders play with it!
The core design goal of the Physical Web is to have it be as quick as possible. The primary use case is to walk up to something (like a vending machine) take your phone out of your pocket and interact with it as quickly as possible. The question them becomes how to do that. The advantage of lockscreen + notifications is that it just shows ups wherever the user is. This is what we're doing on Android w/ Chrome. Putting it 'inside' any app, no matter which one just adds that much more interaction (and user understanding necessary). The lockscreen approach is clearly the fastest way of doing this. Notifications allow the same interaction to occur if the phone is already open (usually the two are strongly related)
Screen 1: showing that number of beacons found. This means that scanning occurs on screen wake up and this will show up ~2 seconds later. It would be a bad UX to ALWAYS show this as many times (right now, most times ;-) it would be empty. It's important that the user will be rewarded if they click on it. Otherwise they will tire and ignore it. Besides, it would take up too much space to always show.
Attached image UX Screen 2 (more info) — 
Screen 2: When tapped, the item in Screen 1 opens to show the first 2 items with a 'show more' link. It is expected that most times there won't be many beacons near the user so showing 1-2 items is more direct. Also, by sorting results by signal strength, even if there are >2, the thing you are standing in front of is likely to be in this list.

Note: the icons shows here are expected to be the favicon which may not always be high quality or look good. The colors also vary (I chose these colors on purpose to not match the homescreen.
I've added 2 UX screens as attachments to this bug. These screens reflect much of the work we have learned in building our versions for Chrome: show something quickly (but only if something is there!) optimise for the first 2 items, show more if needed. I expect there could be lots of discussion so I'm starting simply, gathering a bit of feedback before proceeding in more detail.

I would also appreciate some advice. An email thread (with dozens of people) is rarely an efficient way to discuss UX issues. I'm happy to have feedback but it's very easy to get lost in details. Any suggestions on how the Mozilla culture reviews UX designs without getting lost would be appreciated.
Scott, let me introduce Rob; he is our UX lead on the FirefoxOS project. Rob could you give Scott some feedback here, or forward this on to someone else on your team, thanks!
Flags: needinfo?(rmacdonald)
I've some tech concern about screen 1, 
to show the number of nearby devices may means we need to `scan` nearby devices when we light on the screen.

I've heard that scan BT beacon will cost more power than broadcast the beacon?

ni @jocelyn who implement the GATT protocol on FxOS. Is GATT scan cost more power or it's just at same power consumption level like turn bluetooth enabled and let other device connect to the phone?
Flags: needinfo?(joliu)
Fred, we'd done extensive testing on this. The power required for lighting the screen is well over an order of magnitude more than doing a passive BLE scan. In addition, we only do it for ~10secs and then stop. In all of our tests measuring battery consumption, we've hardly been able to notice the impact (with the assumption that you have bluetooth turned on of course)

Also, for these screens, we never do any GATT connections, we only receive nearby ad packets passively. This is very lightweight and low power.
Switching UX ni? to be triaged by the group.
Flags: needinfo?(rmacdonald) → needinfo?(firefoxos-ux-bugzilla)
Hello Scott and we're glad to have you join us.

I see Harly has already directed you to some of the existing resources. They're not all up to date but feel free to NI us with any questions. 

Although we don't have anyone available to review this in detail, we don't want to block progress on this. So feel free to experiment and innovate as long as it's a developer setting and off by default. Once the feature has had a chance to bake and is on our product roadmap, we'll be able to dedicate someone to work with the team to ensure the experience is in line with the product. At that point we can also look at turning it on by default to collect additional feedback.

Looking at what you and Jan have so far, this is really exciting stuff. The lock screen and utility tray seem like good candidates and deliver that immediacy. Our search bar is getting pretty crowded and is overdue for an optimization so I'd avoid that for now.

I'm not familiar with the standard but here are some items to consider off the top of my head. So please take them with a grain of salt. :)

Screen 1
- Having the scan button permanently occupying a notification space takes up a lot of real estate. With some sort of opt in, could we consider automatically scanning when the phone is awake?
- In your research did the term "Physical Web" make sense to people? I wonder if there's another name like "Stuff nearby" or "Around you". Okay... not sure if those are better but I make my point. Although I suppose "Bluetooth" didn't make much sense initially either.
- Also for this screen, should we consider giving the user a clue as to what is nearby? If it's a Coke machine, I may not feel like interacting with it. If it's a discounted pizza slice... maybe.
- I would also suspect that many segments would find this service disruptive. So the end product would definitely need a user setting. It could be on or off by default depending on the target.

Screen 2
- Again, I'm not familiar with the standard, but are users able to choose what types of devices they want to interact with? In other words, how does this scale? If we're in a world where every cereal box, parking meter or hot dog stand is transmitting something, what does that look like? Do people choose to only receive notifications from certain types of devices? Can they block stuff? Are devices prioritized?

Not sure if this helps but, again, feel free to NI with questions. 

- Rob
Thanks for the feedback Rob! Let me try to answer your questions in order:

Screen 1:
* We're in agreement, the behavior should be exactly how you describe it (but different from Jan's video) This widget should only show up *if* there are beacons nearby. This does mean that there will likely be a small delay from screen on to display of the lockscreen widget but that is much better than permanently taking up space. To show something, have the user click 'scan' and then show nothing would basically train them to never push it again. This is especially true as for the first few months, as beacons will be understandably rare...

* We've been using the term "Physical Web" for over a year now, testing it in talks, demos, and conferences. We've a large amount of press around it as well. What we like about it is that it's fairly non-technical (unlike 'wifi' or 'ethernet') However, we expect that much like wifi, as it starts to be used, people will want a known agreed upon name (and logo) We wouldn't want each browser to call is something different. Short answer: "Physical Web" has been working well for us up to now.

* Good catch on the coke vs pizza display. These two screens are a bit TOO simplistic. The goal is to show  screen 1 where there are LOTS nearby but if you have just 1 or 2, we could go straight to Screen 2. However, I'm still not sure about FFOS UX guidelines and I was worried that taking up too much space would be a frowned upon. Happy to discuss this with you in more detail. But in principle we are in agreement: let's make it easy to pick something from a short list and not force an 'open command' if we can help it.

Screen 2
* Scaling: Initially we'll be thrilled to show the user a single beacon. But clearly as this takes off there will be more. Right now, we go through a proxy service (the Physical Web Server or PWS) which fetches the Title/Description/favicon on behalf of the phone (this saves lots of data/processing and also protect the user from any fingerprinting) The PWS is open source and if if FFOS wishes, you could setup your own. Right now we're ranking based on signal strength. We've found that in most of our scenarios you tend to be in front of something so showing the strongest one first (even if there are 12 visible) often works quite well. However, we *do* expect that we'll do much more in the PWS to sort/rank/filter devices over time. We can even create a more sophisticated browser list, but we'll take that slowly as we learn more.

Sorry, what does NI mean? I'm happy to chat/VC/phone call if that would be quicker. The whole team over here is excited that you folks are considering this. We so want this to be on other platforms, not just Chrome.
(In reply to scott from comment #24)
> Sorry, what does NI mean? I'm happy to chat/VC/phone call if that would be
> quicker. The whole team over here is excited that you folks are considering
> this. We so want this to be on other platforms, not just Chrome.

NI = 'needinfo', it's a way you can draw someone's attention to a bug on Bugzilla.

Regarding UX. In the case that there are loads of beacons (potentially at conferences), is the lockscreen the best place for a user to sift through them? Maybe just showing the number found and then opening a Physical Web app to provide more detail could be another option? This sidesteps the screen real-estate issue.
Good point. Here is the more detailed flow than my screens were able to show:
1) Nothing on home screen if no beacons nearby
2) If <=2 beacons show them on home screen (for a direct tap)
3) If >2 still show the 2 closest with a MORE button
4) MORE takes you away from lockscreen to a fully scrolling app view

Admittedly there is a lot going on there. It's UX optimized to show the thing in front of the user even if there are lots of beacons. A more conservative approach would be:
1) Nothing on home screen if no beacons nearby
2) Show the beacon count as a single lockscreen item
3) Tapping this item navigates to full view for full scrolling list

This means more taps for the user when there is 1 thing but it is much simpler to implement. We could start with this to get the ball rolling, gather feedback and improve over time.
I'm out next week but it sounds the thread is on the right track. 

In the meantime feel free to get in touch with Francis Djabri. He's running a study next week and will likely be quite busy but I've "NI'd" him on this post in case there are any quick questions. Amy Lee (also NI'd) is our visual design lead on lock screen and may also be able to answer your questions.

And forgive my use of Mozilla lingo like "NI". After a while you just internalize it. :)

Thanks!
Rob
Flags: needinfo?(firefoxos-ux-bugzilla)
Flags: needinfo?(fdjabri)
Flags: needinfo?(amlee)
Happy to talk to Francis! We used to work together at Symbian. Francis, I'm happy to turn this into an excuse for breakfast/lunch/coffee (assuming you're at the MTV headquarters?)
(In reply to Fred Lin [:gasolin] from comment #20)
> I've some tech concern about screen 1, 
> to show the number of nearby devices may means we need to `scan` nearby
> devices when we light on the screen.
> 
> I've heard that scan BT beacon will cost more power than broadcast the
> beacon?
> 
> ni @jocelyn who implement the GATT protocol on FxOS. Is GATT scan cost more
> power or it's just at same power consumption level like turn bluetooth
> enabled and let other device connect to the phone?

Sorry for my response here.
IMO, LeScan is not a heavy operation like discovery in classic bluetooth.
Keep scanning when user is using lockscreen seems reasonable to me.

Though I think there should be a way for users to explicitly disable this functionality.
Flags: needinfo?(joliu)
(In reply to scott from comment #28)
> Happy to talk to Francis! We used to work together at Symbian. Francis, I'm
> happy to turn this into an excuse for breakfast/lunch/coffee (assuming
> you're at the MTV headquarters?)

Scott, hi! Great to see you and the Physical Web on Bugzilla - this has always seemed like an ideal feature for Firefox OS. I'm in Georgia this week at a conference but let's meet next week when I'm back in town. I'll send you an email so we can co-ordinate.
Flags: needinfo?(fdjabri)
Hi,


I have an relevant add-on demo that implements similar ideas in this discussion,
Here's the demo video:
https://www.youtube.com/watch?v=cS7_ARcTTQs
and source code:
https://github.com/elin-moco/fxos-addon-beacon-scanner

I put the scan tray at the bottom-right of the screen,
you can enable scanning by swiping it towards top-left, and click on it again to disable,
it's on top of lockscreen and any app (and won't block pop-up dialogs),
so you can easily get to access it whenever you need.

Current features include:
* calculating distance of ibeacons nearby, so you can use it to find stuff put together with ibeacons.
* Supports eddystone-URL, you can open links broadcasted by eddystone beacon.
* Hard-coded in the add-on to launch Moz TPE tour app from http://mzl.tpe, might need to figure out a good way to allow app launching via URLs from beacons.


P.S: The distance calculation and trilateration code is remixed from Jan's repo, thanks Jan for the heavy-lifting work!
Hi! I wanted to make sure this did not remain stale. Since this conversation, Chrome Android & iOS has physical web support. Couple notes from those implementations:

iOS:
- Enabling a widget is cumbersome and the today screen is often ignored, but also a good separator. I think a method that automatically adds a widget would be a good compromise
- iOS lacks any metadata beyond title, that was not fantastic. 

Android:
- Does not show any meta info in notification, but comes turned on (if Bluetooth is on)
- The user receives a push notification when encountering the first beacon. This is fantastic for spreading the physical web.
- Experience once in the physical web section is pretty much the same as within the open source physical web app. 

I think there is opportunity to build upon the metadata shown. Opera pulls in a picture and the navigation of a webpage.
Firefox OS is not being worked on
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → WONTFIX
Flags: needinfo?(amlee)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: