Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 821 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 4.3281. Min Range: -4.3281

Max Act: 4.3281. Min Act: -0.1700

Data Index: 860069 (C4 (Web Text))

Max Activating Token Index: 840

Text #1

Max Range: 4.3281. Min Range: -4.3281

Max Act: 4.2852. Min Act: -0.1700

Data Index: 364949 (C4 (Web Text))

Max Activating Token Index: 991

Text #2

Max Range: 4.3281. Min Range: -4.3281

Max Act: 4.2686. Min Act: -0.1700

Data Index: 376914 (C4 (Web Text))

Max Activating Token Index: 704

Text #3

Max Range: 4.3281. Min Range: -4.3281

Max Act: 4.1957. Min Act: -0.1700

Data Index: 315327 (C4 (Web Text))

Max Activating Token Index: 866

Text #4

Max Range: 4.3281. Min Range: -4.3281

Max Act: 4.2005. Min Act: -0.1700

Data Index: 735183 (C4 (Web Text))

Max Activating Token Index: 961

Text #5

Max Range: 4.3281. Min Range: -4.3281

Max Act: 4.1546. Min Act: -0.1700

Data Index: 1359245 (C4 (Web Text))

Max Activating Token Index: 462

Text #6

Max Range: 4.3281. Min Range: -4.3281

Max Act: 4.1646. Min Act: -0.1700

Data Index: 957161 (C4 (Web Text))

Max Activating Token Index: 918

Text #7

Max Range: 4.3281. Min Range: -4.3281

Max Act: 4.1505. Min Act: -0.1700

Data Index: 892930 (C4 (Web Text))

Max Activating Token Index: 550

Text #8

Max Range: 4.3281. Min Range: -4.3281

Max Act: 4.1710. Min Act: -0.1700

Data Index: 163044 (C4 (Web Text))

Max Activating Token Index: 760

Text #9

Max Range: 4.3281. Min Range: -4.3281

Max Act: 4.1205. Min Act: -0.1700

Data Index: 690874 (C4 (Web Text))

Max Activating Token Index: 941

Text #10

Max Range: 4.3281. Min Range: -4.3281

Max Act: 4.1184. Min Act: -0.1700

Data Index: 1529944 (Python Code)

Max Activating Token Index: 696

Text #11

Max Range: 4.3281. Min Range: -4.3281

Max Act: 4.1184. Min Act: -0.1700

Data Index: 63806 (C4 (Web Text))

Max Activating Token Index: 20

Text #12

Max Range: 4.3281. Min Range: -4.3281

Max Act: 4.1138. Min Act: -0.1700

Data Index: 292899 (C4 (Web Text))

Max Activating Token Index: 979

Text #13

Max Range: 4.3281. Min Range: -4.3281

Max Act: 4.1217. Min Act: -0.1700

Data Index: 81721 (C4 (Web Text))

Max Activating Token Index: 111

Text #14

Max Range: 4.3281. Min Range: -4.3281

Max Act: 4.1257. Min Act: -0.1700

Data Index: 402846 (C4 (Web Text))

Max Activating Token Index: 314

Text #15

Max Range: 4.3281. Min Range: -4.3281

Max Act: 4.1030. Min Act: -0.1700

Data Index: 993085 (C4 (Web Text))

Max Activating Token Index: 361

Text #16

Max Range: 4.3281. Min Range: -4.3281

Max Act: 4.1144. Min Act: -0.1700

Data Index: 495245 (C4 (Web Text))

Max Activating Token Index: 472

Text #17

Max Range: 4.3281. Min Range: -4.3281

Max Act: 4.1062. Min Act: -0.1700

Data Index: 26471 (C4 (Web Text))

Max Activating Token Index: 413

Text #18

Max Range: 4.3281. Min Range: -4.3281

Max Act: 4.0757. Min Act: -0.1700

Data Index: 973100 (C4 (Web Text))

Max Activating Token Index: 48

Text #19

Max Range: 4.3281. Min Range: -4.3281

Max Act: 4.0921. Min Act: -0.1700

Data Index: 1329141 (C4 (Web Text))

Max Activating Token Index: 108