Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1082 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.6861. Min Range: -3.6861

Max Act: 3.6861. Min Act: -0.1700

Data Index: 1240334 (C4 (Web Text))

Max Activating Token Index: 872

Text #1

Max Range: 3.6861. Min Range: -3.6861

Max Act: 3.6007. Min Act: -0.1700

Data Index: 514132 (C4 (Web Text))

Max Activating Token Index: 23

Text #2

Max Range: 3.6861. Min Range: -3.6861

Max Act: 3.5754. Min Act: -0.1700

Data Index: 556617 (C4 (Web Text))

Max Activating Token Index: 27

Text #3

Max Range: 3.6861. Min Range: -3.6861

Max Act: 3.5710. Min Act: -0.1700

Data Index: 83458 (C4 (Web Text))

Max Activating Token Index: 65

Text #4

Max Range: 3.6861. Min Range: -3.6861

Max Act: 3.4984. Min Act: -0.1700

Data Index: 57594 (C4 (Web Text))

Max Activating Token Index: 629

Text #5

Max Range: 3.6861. Min Range: -3.6861

Max Act: 3.5044. Min Act: -0.1700

Data Index: 646664 (C4 (Web Text))

Max Activating Token Index: 583

Text #6

Max Range: 3.6861. Min Range: -3.6861

Max Act: 3.5235. Min Act: -0.1700

Data Index: 486124 (C4 (Web Text))

Max Activating Token Index: 29

Text #7

Max Range: 3.6861. Min Range: -3.6861

Max Act: 3.5034. Min Act: -0.1700

Data Index: 451374 (C4 (Web Text))

Max Activating Token Index: 10

Text #8

Max Range: 3.6861. Min Range: -3.6861

Max Act: 3.5156. Min Act: -0.1700

Data Index: 745800 (C4 (Web Text))

Max Activating Token Index: 724

Text #9

Max Range: 3.6861. Min Range: -3.6861

Max Act: 3.4593. Min Act: -0.1699

Data Index: 60365 (C4 (Web Text))

Max Activating Token Index: 100

Text #10

Max Range: 3.6861. Min Range: -3.6861

Max Act: 3.4720. Min Act: -0.1700

Data Index: 1212322 (C4 (Web Text))

Max Activating Token Index: 22

Text #11

Max Range: 3.6861. Min Range: -3.6861

Max Act: 3.4836. Min Act: -0.1700

Data Index: 61301 (C4 (Web Text))

Max Activating Token Index: 582

Text #12

Max Range: 3.6861. Min Range: -3.6861

Max Act: 3.4872. Min Act: -0.1700

Data Index: 204844 (C4 (Web Text))

Max Activating Token Index: 75

Text #13

Max Range: 3.6861. Min Range: -3.6861

Max Act: 3.4393. Min Act: -0.1700

Data Index: 781880 (C4 (Web Text))

Max Activating Token Index: 89

Text #14

Max Range: 3.6861. Min Range: -3.6861

Max Act: 3.4276. Min Act: -0.1700

Data Index: 950849 (C4 (Web Text))

Max Activating Token Index: 60

Text #15

Max Range: 3.6861. Min Range: -3.6861

Max Act: 3.4304. Min Act: -0.1700

Data Index: 362816 (C4 (Web Text))

Max Activating Token Index: 359

Text #16

Max Range: 3.6861. Min Range: -3.6861

Max Act: 3.4386. Min Act: -0.1700

Data Index: 604678 (C4 (Web Text))

Max Activating Token Index: 105

Text #17

Max Range: 3.6861. Min Range: -3.6861

Max Act: 3.4066. Min Act: -0.1700

Data Index: 331372 (C4 (Web Text))

Max Activating Token Index: 48

Text #18

Max Range: 3.6861. Min Range: -3.6861

Max Act: 3.4143. Min Act: -0.1700

Data Index: 555459 (C4 (Web Text))

Max Activating Token Index: 120

Text #19

Max Range: 3.6861. Min Range: -3.6861

Max Act: 3.4029. Min Act: -0.1700

Data Index: 559221 (C4 (Web Text))

Max Activating Token Index: 935