Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 634 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.4956. Min Range: -3.4956

Max Act: 3.4956. Min Act: -0.1700

Data Index: 555128 (C4 (Web Text))

Max Activating Token Index: 66

Text #1

Max Range: 3.4956. Min Range: -3.4956

Max Act: 3.4704. Min Act: -0.1700

Data Index: 973626 (C4 (Web Text))

Max Activating Token Index: 740

Text #2

Max Range: 3.4956. Min Range: -3.4956

Max Act: 3.4015. Min Act: -0.1700

Data Index: 711341 (C4 (Web Text))

Max Activating Token Index: 467

Text #3

Max Range: 3.4956. Min Range: -3.4956

Max Act: 3.3805. Min Act: -0.1700

Data Index: 1209213 (C4 (Web Text))

Max Activating Token Index: 733

Text #4

Max Range: 3.4956. Min Range: -3.4956

Max Act: 3.3736. Min Act: -0.1700

Data Index: 155692 (C4 (Web Text))

Max Activating Token Index: 844

Text #5

Max Range: 3.4956. Min Range: -3.4956

Max Act: 3.3904. Min Act: -0.1700

Data Index: 708981 (C4 (Web Text))

Max Activating Token Index: 12

Text #6

Max Range: 3.4956. Min Range: -3.4956

Max Act: 3.3769. Min Act: -0.1700

Data Index: 795285 (C4 (Web Text))

Max Activating Token Index: 733

Text #7

Max Range: 3.4956. Min Range: -3.4956

Max Act: 3.3927. Min Act: -0.1700

Data Index: 813670 (C4 (Web Text))

Max Activating Token Index: 700

Text #8

Max Range: 3.4956. Min Range: -3.4956

Max Act: 3.3728. Min Act: -0.1700

Data Index: 439822 (C4 (Web Text))

Max Activating Token Index: 889

Text #9

Max Range: 3.4956. Min Range: -3.4956

Max Act: 3.3850. Min Act: -0.1700

Data Index: 1194497 (C4 (Web Text))

Max Activating Token Index: 701

Text #10

Max Range: 3.4956. Min Range: -3.4956

Max Act: 3.3581. Min Act: -0.1700

Data Index: 71730 (C4 (Web Text))

Max Activating Token Index: 57

Text #11

Max Range: 3.4956. Min Range: -3.4956

Max Act: 3.3382. Min Act: -0.1700

Data Index: 959336 (C4 (Web Text))

Max Activating Token Index: 111

Text #12

Max Range: 3.4956. Min Range: -3.4956

Max Act: 3.3559. Min Act: -0.1700

Data Index: 507479 (C4 (Web Text))

Max Activating Token Index: 43

Text #13

Max Range: 3.4956. Min Range: -3.4956

Max Act: 3.3369. Min Act: -0.1700

Data Index: 691293 (C4 (Web Text))

Max Activating Token Index: 267

Text #14

Max Range: 3.4956. Min Range: -3.4956

Max Act: 3.3657. Min Act: -0.1700

Data Index: 629920 (C4 (Web Text))

Max Activating Token Index: 233

Text #15

Max Range: 3.4956. Min Range: -3.4956

Max Act: 3.3244. Min Act: -0.1700

Data Index: 617808 (C4 (Web Text))

Max Activating Token Index: 740

Text #16

Max Range: 3.4956. Min Range: -3.4956

Max Act: 3.3238. Min Act: -0.1700

Data Index: 44146 (C4 (Web Text))

Max Activating Token Index: 742

Text #17

Max Range: 3.4956. Min Range: -3.4956

Max Act: 3.3287. Min Act: -0.1700

Data Index: 1160821 (C4 (Web Text))

Max Activating Token Index: 650

Text #18

Max Range: 3.4956. Min Range: -3.4956

Max Act: 3.3580. Min Act: -0.1700

Data Index: 117255 (C4 (Web Text))

Max Activating Token Index: 50

Text #19

Max Range: 3.4956. Min Range: -3.4956

Max Act: 3.3588. Min Act: -0.1700

Data Index: 501687 (C4 (Web Text))

Max Activating Token Index: 838