Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.9982. Min Range: -1.9982

Max Act: 1.9982. Min Act: -0.0000

Data Index: 1461734 (The Pile)

Max Activating Token Index: 124

Text #1

Max Range: 1.9982. Min Range: -1.9982

Max Act: 1.8961. Min Act: -0.0000

Data Index: 141709 (The Pile)

Max Activating Token Index: 809

Text #2

Max Range: 1.9982. Min Range: -1.9982

Max Act: 1.7822. Min Act: -0.0000

Data Index: 891976 (The Pile)

Max Activating Token Index: 438

Text #3

Max Range: 1.9982. Min Range: -1.9982

Max Act: 1.7595. Min Act: -0.0000

Data Index: 1547434 (The Pile)

Max Activating Token Index: 465

Text #4

Max Range: 1.9982. Min Range: -1.9982

Max Act: 1.6616. Min Act: -0.0000

Data Index: 319115 (The Pile)

Max Activating Token Index: 556

Text #5

Max Range: 1.9982. Min Range: -1.9982

Max Act: 1.8462. Min Act: -0.0000

Data Index: 563127 (The Pile)

Max Activating Token Index: 115

Text #6

Max Range: 1.9982. Min Range: -1.9982

Max Act: 1.6521. Min Act: -0.0000

Data Index: 973955 (The Pile)

Max Activating Token Index: 1009

Text #7

Max Range: 1.9982. Min Range: -1.9982

Max Act: 1.8395. Min Act: -0.0000

Data Index: 1094078 (The Pile)

Max Activating Token Index: 435

Text #8

Max Range: 1.9982. Min Range: -1.9982

Max Act: 1.7127. Min Act: -0.0000

Data Index: 546916 (The Pile)

Max Activating Token Index: 377

Text #9

Max Range: 1.9982. Min Range: -1.9982

Max Act: 1.7059. Min Act: -0.0000

Data Index: 870863 (The Pile)

Max Activating Token Index: 452

Text #10

Max Range: 1.9982. Min Range: -1.9982

Max Act: 1.7011. Min Act: -0.0000

Data Index: 1327241 (The Pile)

Max Activating Token Index: 310

Text #11

Max Range: 1.9982. Min Range: -1.9982

Max Act: 1.6066. Min Act: -0.0000

Data Index: 1105266 (The Pile)

Max Activating Token Index: 330

Text #12

Max Range: 1.9982. Min Range: -1.9982

Max Act: 1.6937. Min Act: -0.0000

Data Index: 715596 (The Pile)

Max Activating Token Index: 866

Text #13

Max Range: 1.9982. Min Range: -1.9982

Max Act: 1.6835. Min Act: -0.0000

Data Index: 331461 (The Pile)

Max Activating Token Index: 610

Text #14

Max Range: 1.9982. Min Range: -1.9982

Max Act: 1.7939. Min Act: -0.0000

Data Index: 292125 (The Pile)

Max Activating Token Index: 725

Text #15

Max Range: 1.9982. Min Range: -1.9982

Max Act: 1.5717. Min Act: -0.0000

Data Index: 1584212 (The Pile)

Max Activating Token Index: 819

Text #16

Max Range: 1.9982. Min Range: -1.9982

Max Act: 1.5731. Min Act: -0.0000

Data Index: 1682709 (The Pile)

Max Activating Token Index: 517

Text #17

Max Range: 1.9982. Min Range: -1.9982

Max Act: 1.6723. Min Act: -0.0000

Data Index: 175982 (The Pile)

Max Activating Token Index: 693

Text #18

Max Range: 1.9982. Min Range: -1.9982

Max Act: 1.6432. Min Act: -0.0000

Data Index: 359508 (The Pile)

Max Activating Token Index: 590

Text #19

Max Range: 1.9982. Min Range: -1.9982

Max Act: 1.6532. Min Act: -0.0000

Data Index: 1878595 (The Pile)

Max Activating Token Index: 442