Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 305 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 4.2828. Min Range: -4.2828

Max Act: 4.2828. Min Act: -0.1700

Data Index: 378188 (C4 (Web Text))

Max Activating Token Index: 761

Text #1

Max Range: 4.2828. Min Range: -4.2828

Max Act: 4.2351. Min Act: -0.1700

Data Index: 415346 (C4 (Web Text))

Max Activating Token Index: 121

Text #2

Max Range: 4.2828. Min Range: -4.2828

Max Act: 4.2045. Min Act: -0.1700

Data Index: 308383 (C4 (Web Text))

Max Activating Token Index: 602

Text #3

Max Range: 4.2828. Min Range: -4.2828

Max Act: 4.1805. Min Act: -0.1700

Data Index: 1304972 (C4 (Web Text))

Max Activating Token Index: 572

Text #4

Max Range: 4.2828. Min Range: -4.2828

Max Act: 4.1760. Min Act: -0.1700

Data Index: 670831 (C4 (Web Text))

Max Activating Token Index: 908

Text #5

Max Range: 4.2828. Min Range: -4.2828

Max Act: 4.1614. Min Act: -0.1700

Data Index: 1125130 (C4 (Web Text))

Max Activating Token Index: 710

Text #6

Max Range: 4.2828. Min Range: -4.2828

Max Act: 4.1375. Min Act: -0.1700

Data Index: 907714 (C4 (Web Text))

Max Activating Token Index: 589

Text #7

Max Range: 4.2828. Min Range: -4.2828

Max Act: 4.1212. Min Act: -0.1700

Data Index: 596492 (C4 (Web Text))

Max Activating Token Index: 731

Text #8

Max Range: 4.2828. Min Range: -4.2828

Max Act: 4.1236. Min Act: -0.1700

Data Index: 379487 (C4 (Web Text))

Max Activating Token Index: 116

Text #9

Max Range: 4.2828. Min Range: -4.2828

Max Act: 4.1249. Min Act: -0.1700

Data Index: 226100 (C4 (Web Text))

Max Activating Token Index: 937

Text #10

Max Range: 4.2828. Min Range: -4.2828

Max Act: 4.1379. Min Act: -0.1700

Data Index: 19339 (C4 (Web Text))

Max Activating Token Index: 58

Text #11

Max Range: 4.2828. Min Range: -4.2828

Max Act: 4.1047. Min Act: -0.1700

Data Index: 1025828 (C4 (Web Text))

Max Activating Token Index: 760

Text #12

Max Range: 4.2828. Min Range: -4.2828

Max Act: 4.1048. Min Act: -0.1700

Data Index: 346714 (C4 (Web Text))

Max Activating Token Index: 133

Text #13

Max Range: 4.2828. Min Range: -4.2828

Max Act: 4.0998. Min Act: -0.1700

Data Index: 843691 (C4 (Web Text))

Max Activating Token Index: 818

Text #14

Max Range: 4.2828. Min Range: -4.2828

Max Act: 4.0996. Min Act: -0.1700

Data Index: 1306102 (C4 (Web Text))

Max Activating Token Index: 998

Text #15

Max Range: 4.2828. Min Range: -4.2828

Max Act: 4.0814. Min Act: -0.1700

Data Index: 227663 (C4 (Web Text))

Max Activating Token Index: 538

Text #16

Max Range: 4.2828. Min Range: -4.2828

Max Act: 4.0705. Min Act: -0.1700

Data Index: 666444 (C4 (Web Text))

Max Activating Token Index: 1009

Text #17

Max Range: 4.2828. Min Range: -4.2828

Max Act: 4.0916. Min Act: -0.1700

Data Index: 582206 (C4 (Web Text))

Max Activating Token Index: 117

Text #18

Max Range: 4.2828. Min Range: -4.2828

Max Act: 4.0728. Min Act: -0.1700

Data Index: 1235171 (C4 (Web Text))

Max Activating Token Index: 507

Text #19

Max Range: 4.2828. Min Range: -4.2828

Max Act: 4.0934. Min Act: -0.1700

Data Index: 877287 (C4 (Web Text))

Max Activating Token Index: 958