Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 211 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 2.9143. Min Range: -2.9143

Max Act: 2.9143. Min Act: -0.1700

Data Index: 1098878 (C4 (Web Text))

Max Activating Token Index: 823

Text #1

Max Range: 2.9143. Min Range: -2.9143

Max Act: 2.8059. Min Act: -0.1700

Data Index: 852561 (C4 (Web Text))

Max Activating Token Index: 506

Text #2

Max Range: 2.9143. Min Range: -2.9143

Max Act: 2.7791. Min Act: -0.1700

Data Index: 110844 (C4 (Web Text))

Max Activating Token Index: 662

Text #3

Max Range: 2.9143. Min Range: -2.9143

Max Act: 2.7652. Min Act: -0.1700

Data Index: 553421 (C4 (Web Text))

Max Activating Token Index: 181

Text #4

Max Range: 2.9143. Min Range: -2.9143

Max Act: 2.7585. Min Act: -0.1700

Data Index: 154415 (C4 (Web Text))

Max Activating Token Index: 786

Text #5

Max Range: 2.9143. Min Range: -2.9143

Max Act: 2.7775. Min Act: -0.1700

Data Index: 268725 (C4 (Web Text))

Max Activating Token Index: 232

Text #6

Max Range: 2.9143. Min Range: -2.9143

Max Act: 2.7423. Min Act: -0.1700

Data Index: 1303103 (C4 (Web Text))

Max Activating Token Index: 519

Text #7

Max Range: 2.9143. Min Range: -2.9143

Max Act: 2.7410. Min Act: -0.1700

Data Index: 1339689 (C4 (Web Text))

Max Activating Token Index: 574

Text #8

Max Range: 2.9143. Min Range: -2.9143

Max Act: 2.7557. Min Act: -0.1700

Data Index: 38279 (C4 (Web Text))

Max Activating Token Index: 387

Text #9

Max Range: 2.9143. Min Range: -2.9143

Max Act: 2.7565. Min Act: -0.1700

Data Index: 388297 (C4 (Web Text))

Max Activating Token Index: 193

Text #10

Max Range: 2.9143. Min Range: -2.9143

Max Act: 2.7315. Min Act: -0.1700

Data Index: 618489 (C4 (Web Text))

Max Activating Token Index: 475

Text #11

Max Range: 2.9143. Min Range: -2.9143

Max Act: 2.7463. Min Act: -0.1700

Data Index: 168327 (C4 (Web Text))

Max Activating Token Index: 931

Text #12

Max Range: 2.9143. Min Range: -2.9143

Max Act: 2.7287. Min Act: -0.1700

Data Index: 22950 (C4 (Web Text))

Max Activating Token Index: 6

Text #13

Max Range: 2.9143. Min Range: -2.9143

Max Act: 2.7046. Min Act: -0.1700

Data Index: 24150 (C4 (Web Text))

Max Activating Token Index: 745

Text #14

Max Range: 2.9143. Min Range: -2.9143

Max Act: 2.7097. Min Act: -0.1700

Data Index: 523556 (C4 (Web Text))

Max Activating Token Index: 444

Text #15

Max Range: 2.9143. Min Range: -2.9143

Max Act: 2.7138. Min Act: -0.1700

Data Index: 745666 (C4 (Web Text))

Max Activating Token Index: 435

Text #16

Max Range: 2.9143. Min Range: -2.9143

Max Act: 2.7020. Min Act: -0.1700

Data Index: 782128 (C4 (Web Text))

Max Activating Token Index: 786

Text #17

Max Range: 2.9143. Min Range: -2.9143

Max Act: 2.7064. Min Act: -0.1700

Data Index: 205382 (C4 (Web Text))

Max Activating Token Index: 116

Text #18

Max Range: 2.9143. Min Range: -2.9143

Max Act: 2.7220. Min Act: -0.1700

Data Index: 890292 (C4 (Web Text))

Max Activating Token Index: 228

Text #19

Max Range: 2.9143. Min Range: -2.9143

Max Act: 2.6825. Min Act: -0.1700

Data Index: 266549 (C4 (Web Text))

Max Activating Token Index: 876